When processing a log file not all the hits should be included in the generated statistics. Log files can sometimes span multiple months and/or contain hits that you will have no interest in (for example requests to retrieve css files, javascript files, image files, the favicon.ico, the robots.txt file). As such it is desirable to be able to filter the hits to only include in the statistics those that you have an interest in. Hit filtering allows you to do that.

Hit filtering will only occur if a hit filter is provided by specifying a class in property:

The hit filter implementation must implement interface: org.polliwog.filters.HitFilter. For each hit method: accept(org.polliwog.data.Hit) is called, if the method returns true then the hit is added to the statistics, otherwise it is filtered, i.e. rejected.

A basic hit filtering implementation is provided with polliwog. It is implemented by class: org.polliwog.filters.BasicHitFilter. To make polliwog use this filter use value:
for the: hitFilterClass property.

The basic hit filter uses a xml file to configure a number of rules that it applies to the hit to determine whether it should be filtered.

Each rule is applied sequentially. Each rule can define whether the hit should be accepted or rejected (which correspond to whether the accept returns true or false).

The xml file has the following elements/attributes:

XML Definition Help The Children column shows the child elements that can be used within the specified element. Child elements can appear in any order (there is no enforcement via a DTD). A + after the element name indicates that at least 1 child element with that name must be present. A * after the element name indicates that 0 or more elements can be present. A ? after the element name indicates that either 1 or no elements with that name can be present. If no symbol is provided after the name then one element must be provided. The Attributes column shows the attribute that can be used on the specified element. Attribute definitions are defined as: name(value_type,required|optional), where value_type is one of: string - A string value, this can be anything. integer - An integer value. class - A fully qualified classname. enum{values} - A specific value, one or more of those given in the brackets, which will be comma-separated. boolean - Either true or false. Required and optional are represented as: R and O respectively.
Name	Root	Children	Parent(s)	Attributes	Description
hit-filter	Y	rule+	NONE	NONE	The root element, each child rule element defines a rule that should be applied to a hit.
rule	N	ANY	hit-filter	type(string,R) action(string,R)	Defines a rule to be applied to a hit. The type attribute defines the type of rule that is created, the following values are supported: josql - Creates an instance of org.polliwog.filters.JoSQLRule, see JoSQL Rules for more details. date - Creates an instance of org.polliwog.filters.DateRule, see Date Rules for more details. url - Creates an instance of org.polliwog.filters.URLRule, see URL Rules for more details. Depending upon the type of rule created there may be extra attributes needed for the rule element, see the relevant rule for details. The action attribute can be either: accept or reject depending upon whether you want to accept or reject the hit if the rule matches.

A JoSQL rule is created when the the type attribute on a rule element is: josql. An instance of: org.polliwog.filters.JoSQLRule is created.

A josql rule uses a  JoSQL WHERE clause to perform the filtering (you do not need to provide the WHERE keyword). The WHERE clause should be placed as the content of the rule element.

Instances of Hit will be passed to the rule and the WHERE clause applied. If the WHERE clause evaluates to true then the hit is accepted or rejected according to the action attribute.

The functions from JoSQLFunctionHandler are available for use in the WHERE clause.
It should be noted that whilst the Hit object can contain a large amount of information at the point where the filter rules are applied only non-derived information (i.e. information that is available in the log file) has a value. For instance the site area, hit page and visit summary won't have a value so they should not be used in the WHERE clause.

In general, you should use the following accessors in the WHERE clause:

• date
• size
• userAgent
• request
• status
• pageType
• requestURI
• path
• requestParameters
• requestMethod
• hostname
• request
• referer

Only accept hits for the current month.
Reject all hits from the 192.168.0.X ip address range.
Reject hits for image files.
Reject 404 hits that come from a post method where the size is greater than 10000 bytes.

A date rule is created when the the type attribute on a rule element is: date. An instance of: org.polliwog.filters.DateRule is created. A date rule will filter hits based on the date that the hit occurred.

The date rule uses the following extra optional attributes, which must be specified on the rule element, to initialize itself (the value in brackets indicates the type of value that should be specified):

• currentMonth(boolean) - when specified with a true value only hits with a date in the current month will be accepted/rejected. The time between 00:00:000 on the 1st of the calendar month to: 23:59:999 on the last day of the calendar month is considered to be the current month.
• currentWeek(boolean) - when specified with a true value only hits with a date in the current week will be accepted/rejected. The time between 00:00:000 on the first day of the current week to 23:59:999 6 days later. In general (for most locales) this will mean either between: Monday 00:00:000 - Sunday 23:59:999 or: Sunday 00:00:000 - Saturday 23:59:999 either way, the time period used will span 7 days (minus 1 millisecond).
  
  Note: the first day of the week is determined by calling:  getFirstDayOfWeek(), the week is then taken as being seven days after that day.
• today(boolean) - when specified with a true value only hits for the current date will be accepted/rejected.
• after(string) - indicates that only hits after the date specified should be accepted/rejected. The default date format is: dd/MMM/yyyy but this can be overridden by using the format attribute.
• before(string) - indicates that only hits before the date specified should be accepted/rejected. The default date format is: dd/MMM/yyyy but this can be overridden by using the format attribute.
• format(string) - indicates that the format specified should be used instead of the default. The format should be suitable for use with a  SimpleDateFormat object.
• month(string) - when specified only hits for the specified month (for the current year) will be accepted/rejected. The month value should be one of the standard three letter acronyms for your locale, i.e. Jan, Feb, Mar, Apr etc.

Only accept hits for the current month.

Reject hits not between 3rd April 2007 and 23rd April 2007. Note this is the same as reversing the dates and using an action value of: accept.

Accept hits in June.

A url rule is created when the the type attribute on a rule element is: url. An instance of: org.polliwog.filters.URLRule is created. A url rule filters based on the requested url for the hit.

The url rule uses the following extra optional attributes, which must be specified on the rule element, to initialize itself (the value in brackets indicates the type of value that should be specified):

• startsWith(string) - when specified indicates a value that the url should start with for the hit to be filtered.
• endsWith(string) - when specified indicates a value that the url should end with for the hit to be filtered.
• contains(string) - when specified indicates a value that the url should contain for the hit to be filtered.
• ignoreCase(boolean) - when specified with a true value it indicates that comparisons should be case-insensitive

Note: request urls start with /.

Reject hits for the admin part of the site.

Only accept .php hits.