Content Filter Profiles

Filtering requests based on their content

Overview

Content Filtering is the final stage of traffic processing. This fulfills the role of a traditional WAF, which is to examine the content of a request for specific signatures and take specific actions when matches are found.

A Content Filter Profile associates signatures (Content Filter Rules) with specific actions, and includes some other parameters as well.

Circumstances where content filtering will not occur

A request will not be subjected to content filtering if any of these are true:

• It was blocked during a previous stage of processing.

• It triggered a Skip action in a Global Filter, or Bypass in the ACL Profile.

• Its destination URL matches a Security Policy which does not have an active Content Filter Profile assigned to it.

As discussed below, there are also several stages within the content filtering process where some or all of a request might be exempted from completing the process.

Usage within applications and APIs

A Content Filter Profile is assigned to paths/URLs within applications and APIs via Security Policies.

When a request is received, the system checks its destination URL, and uses the Profile that best matches it.

Out of the box, the system includes a default Content Filter Profile. It can be edited, but it cannot be deleted. It is used for all URLs where no other Profile has been assigned.

Administration

The main page lists all current Content Filter Profiles.

The administration (addition/deletion/editing/versioning) of Profiles follows the conventions described here.

The Content Filtering process

Content filtering is a multi-step process:

1. Data Masking: sensitive data in the request is masked, so that it does not appear in traffic logs.

2. Content Type Checking: if one or more Restrict content type settings are enabled, they are enforced.

3. Allowlisting: if the request has one or more specified tags in the Ignore list, it is exempted from further processing.

4. Content Filtering: several forms of content limits are checked. The request's parameters (headers, cookies, and arguments) are then examined. Tags are added to the request based on Content Filter Rules.

5. Tag Evaluation: the request's tags are evaluated for reporting and/or action purposes.

Each step is described in more detail below. For context, here is a graphic showing the controls/sections within the UI that are used in each step:

Click to expand this image.

And here is a flowchart of the process described below, with captions along the top representing the relevant steps:

Step 1: Data Masking

For parameters listed in the Parameter Properties list with the Mask? option enabled, their values are replaced by hashes of these values with the Masking seed.

Step 2: Content Type Checking

If the Ignore Body setting is not enabled, and one or more Restrict content type settings are enabled, the request is evaluated against them. If the request cannot be parsed according to one of the specified types, the Action will be executed.

Step 3: Allowlisting

Before content filtering is performed, the tags that are already attached to the request are evaluated. (These tags could be the result of Global Filters, Rate Limit Rules, or Flow Control Policies.)

If any tag is found in the Ignore list, the request is not subjected to content filtering (i.e., Step 4 and Step 5 below do not occur). This allows certain requests to be allowlisted (for example, if they are from a trusted source), avoiding the overhead of unnecessary processing.

Step 4: Content Filtering

Starting with the headers, each parameter (each header, cookie, and arg) of the request is processed. Each parameter and its value are compared to the entries in the Parameter Properties list: the entries in its All section and also the relevant parameter-type section (either Headers, Cookies, or Arguments).

The Parameter Properties list allows the following to be defined:

• The parameters, if any, whose values should be masked.

• The parameters, if any, whose values should be restricted to regex definitions.

• The parameters, if any, which should be exempted from evaluation against specific Content Filter Rules.

For each parameter, the processing occurs in the order described below, until one of the following happens:

• the parameter is exempted from further inspection

• the Action is triggered (which also terminates the processing of the request)

• the parameter completes its processing

At that point, the next parameter is processed, until none remain.

Parameter processing in detail

• The relevant Max Count limit is checked (excluding decoded content). If it is active and the parameter exceeds the limit, the Action is triggered.

• The relevant Max Length limit is checked (excluding decoded content). If it is active and the parameter exceeds the limit, the Action is triggered.

• If the Ignore Alphanumeric input option is enabled, and all the parameter's content is alphanumeric, this parameter is exempted from further inspection.

• If one or more entries have a definition for Parameter that matches the parameter's name, the entry that most closely matches it is selected. If an entry was selected, and the parameter's value does not match the entry's Matching Value, and the entry's Restrict? option is enabled, the Action is triggered.

• The parameter is evaluated against the Content Filter Rules. Link11 WAAP iterates through the Rules, and for each one:

  • If the parameter matched an entry (in both Parameter and Matching Value), and any of the Rule's tags are listed in the entry's Ignore Content Filter Tags, this Rule is skipped and its tag(s) are not attached.

  • Otherwise, the Rule's Match string is compared to the parameter's value. If a match is found, the Rule's tags are attached to the request.

• This parameter has now completed its processing, and the next parameter in the request is selected.

A parameter can be exempted from all Content Filtering Rules by including cf-all in its Ignore Content Filter Tags.

When the Action is triggered, it is executed immediately, and no further content filtering occurs for that request. Therefore, if a request contains more than one type of attack, one will appear in the logs while the remainder will not.

Step 5: Tag Evaluation

All the request's tags, including the list of new tags accumulated in the previous step from Content Filter Rule evaluation, are now compared to the Report and Active tag lists.

During this process, a request's tags are divided into two categories:

• Specific tags produced by individual rules (and designated with the Rule name: cf-rule-id-XXX)

• General tags (cf-all and all other tags that are not specific)

Specific tags are processed first. This allows admins to set up general configurations for threat categories, while still being able to make exceptions for how specific Content Filter Rules are handled.

The request's list of tags are processed as follows:

1. All specific tags are compared to the Report list. For each tag that is found there, the relevant Content Filter Rule name will be added to the Events Log's Monitor reason field for the request.

2. All general tags are compared to the Report list. For each tag that is found there, the relevant Content Filter Rule name will be added to the Events Log's Monitor reason field for the request.

3. If any of the specific tags are found in the Active list, the Action is executed, and processing ceases.

4. If any of the general tags are found in the Active list, the Action is executed, and processing ceases.

Note that in the UI, the Ignore list appears next to the Active and Report lists. However, the Ignore list is processed earlier in the content filtering process (see Step 3: Allowlisting, above). If any of a request's tags match one in the Ignore list, that request is exempted from content filtering.

An entry in Ignore always "wins". It is dangerous to put anything other than specific tags into it.

Configuring a Positive Security model

Content Filter Profiles can be used as part of a positive security model (where by default, all requests are rejected, except for those identified as being legitimate).

This can be done by ensuring that each parameter (header, argument, or cookie) that could appear within a request is listed in the Parameter Properties list, with an appropriate Matching Value, and with the Restrict? option enabled.

As described above, during processing, each parameter in the incoming request will be evaluated according to the applicable Matching Value.

• If it matches, this parameter will pass the test.

• If it does not match, the Restrict? setting will cause the Action to be executed.

If a positive security model is correctly implemented, with all possible parameters appropriately defined in the list of Parameter Properties, then Step 5: Tag Evaluation would seem to be irrelevant. Invalid requests will trigger the Action before the Active and Report lists are evaluated, and only valid ones would remain when these lists are evaluated; thus, in theory, these lists could be left empty. However, to compensate for potential errors or omissions, it is still wise to include general high-risk tags (such as the cf-rule-risk:5) anyway.

Components

A Content Filter Profile consists of the following:

• Settings for parsing the request (Restrict content type, Decoding, Ignore Alphanumeric Input, Ignore Body)

• Settings for processing the request (Action, Tags)

• Content filtering parameters (the bottom section of the UI, referred to below as the Parameter Properties section)

• Parameters for Step 3: Allowlisting (the Ignore list) and Step 5: Tag Evaluation (Active and Report lists)

• Masking parameters for concealing sensitive information in logs and analytics (Masking seed, and Mask? setting for individual parameters)

• General administration parameters (Name, Description)

Parameters

Global Parameters

In this context, "global" means "applying to the entire request."

Name

A name for this Profile, to be used within the interface. A system tag (shown below the Tags field) will include it as well.

Description

Information about this Profile, for use within the interface.

Masking seed

An admin-defined string for salting the hash when masking private data. See discussion of the Mask? field, below.

Action

The action that can be executed under the various circumstances described above in The Content Filtering Process.

Tags

A list of one or more tags, separated by spaces. Whenever this Content Filter Profile is applied to a request, these tags will appear in the traffic logs.

In addition to these admin-defined tags, the system also shows some system tags that will be attached as well.

Ignore Alphanumeric Input

When this is selected, this Content Filter Profile will not inspect requests that only contain alphanumeric characters. This reduces computational overhead by not evaluating alphanumeric requests. (Hostile requests such as SQLi, XSS, etc., will contain some non-alphanumeric characters.)

Ignore Body

When this is selected, this Content Filter Profile will not inspect the body of the request.

Restrict content type

If Ignore Body is not selected, and one or more Content Types are checked, L11WAAP will expect the request to conform to one of them. If the body cannot be parsed according to one of the specified types, the Action will be executed. If no Types are checked, no restrictions are enforced.

This setting can be a useful way to easily block a variety of attacks. For example, when "Multipart Form" is not selected, a user cannot upload any files, thus preventing malware uploads.

Decoding

Which decoding standard(s) should be used.

When one or more decoding standards are not applicable, they should be unselected here. This will reduce processing time and overhead.

Tag Processing Settings

This section defines what happens to a request, depending on its tags.

The Ignore list

This list is evaluated before content filtering occurs, as described above in Step 3.

The interface has two separate places where a tag can be configured to be ignored: here in this section's Ignore column, and also in the Parameter Properties (discussed below). The setting here is global, and will apply to all tags attached to the request. The settings in the Parameter Properties list apply only to the evaluation of specific parameters.

The Active and Report lists

These lists are evaluated after content filtering rules have been evaluated. See the description above of Step 5.

If a potential tag is not included in the Active or Report lists, it is effectively in Ignore mode. That tag cannot result in the request being blocked, regardless of the severity of the threat signature (i.e., the Content Filter Rule) that produced the tag. Along with ensuring that all potential tags are properly configured, it is also recommended that each Content Filter Rule has an appropriate Risk Level defined, and that each Content Filter Profile has the highest risk-level tags (e.g., cf-rule-risk:5, cf-rule-id:libinjection-sqli, andcf-rule-id:libinjection-xss) included in the Active mode. This ensures that the highest-risk requests will still be blocked.

Parameter Properties

This is the bottom section of the interface. It defines how L11WAAP processes individual parameters within a request.

Max length

The maximum allowable length of the value for this parameter type (header, cookie, or arg).

Max length limit active mode

When enabled, Max length is enforced.

Max count

The maximum allowable number of this parameter type (headers, cookies, or args).

Max count limit active mode

When enabled, Max count is enforced.

Parameter

The parameter whose value will be compared to the Matching Value. This can be provided as a specific Name (e.g., sessionid), or as a Regex to match multiple parameters (e.g., user_.+). Note that a Name will be marked with ABC, while a Regex will be marked with <>.

Matching Value

A regex pattern. If a parameter's value matches it, the Ignore Content Filter Tags become relevant. If it does not match, the Restrict? option becomes relevant.

Restrict?

If a parameter does not match the Matching Value, and Restrict? is selected, then the Action is executed.

Mask?

Some requests might contain private data which should not be saved to a traffic log. Parameters which match the Matching Value and for which Mask? is set will be masked / hashed when they are written to the logs, salted with the Matching Seed from the global settings.

Ignore Content Filter Tags

A parameter whose value matches its Matching Value, or which does not match but Restrict? is not enabled, will be evaluated against the Content Filter Rules. For each Rule that it matches, that Rule's tags will be attached. Sometimes it is desirable to exempt a parameter from the effects of certain Rules. For example, some Rules filter out special characters; if a parameter can legitimately contain these characters, it would make sense to exempt that parameter from those specific filters. To do this, add the tags from those Rules to this list. These tags will then be ignored for this parameter.

Properties are defined in the same way for Headers, Cookies, and Arguments within their respective tabs.