Rate Limiting

Controlling the rates of incoming requests

Rate limiting restricts the rates at which traffic sources can send requests. When a limit is exceeded, the defined action is enforced.

Note: the example Rate Limit rule shown here is very permissive. In production use, stricter parameters will usually be chosen.

Rate Limit Management

Admins can create new Rate Limit rules, as discussed below. Admins can also edit the rules that are included out of the box in a new deployment.

Some rules are provided and maintained by Reblaze, and are read-only. These are designated by the Reblaze icon.

Out of the box, Reblaze's Rate Limit rules are named with a prefix of RL (for Rate Limit) or RLP (a Rate Limit using the "Pair with" attribute, discussed below). This naming convention is recommended when creating/editing new rules, but it is not mandatory.

It is also recommended that Rule names are descriptive of their contents. For example, a Rule that limits 50 requests per username, per 60 seconds, could be named "RL 50/60s username". This makes administration straightforward, and also clarifies events in the traffic logs.

Some of the Reblaze-maintained Rate Limit rules are templates (designated by the prefix RLT or RLPT). If you do not wish to see them on this page, they can be hidden by selecting the Hide Templates checkbox on the upper right. These templates are used by the Create New Site wizard, and can also be assigned to Linked URLs by admins.

Rate Limit Rule Administration

Existing rules are listed on the page shown above. To edit or delete a rule, select the "edit" button at the end of its listing.

To add a new rule, select the + button. The following window will appear.

After defining a new Rate Limit, select the Save button at the top of the window. An additional section (Links to URLs) will then appear at the bottom of the window. This will be discussed later.

Defining a Rate Limit Rule

Each Rate Limit Rule consists of the following types of parameters:

Type	Values
Meta	A name and description for the Rule.
Rate Limit	Defines the maximum allowable number of events within a certain time period.
Event Criteria	Specifies which events will be counted toward the Rate Limit.
Action	Specifies what will happen when the Rate Limit is exceeded.
Assignments	The resources/locations/URLs within the web application where the Rate Limit Rule will be enforced.

Each is discussed further below.

Meta Parameters

Field	Value
Name	A name for this Rule. It will be used within the Reblaze interface, and will also be used to create a Tag that will be assigned to requests which trigger the Rate Limit.
Description	A description of this Rule which will be used within the Reblaze interface.

Rate Limit

Field	Value
Limit	The maximum number of allowable events within the specified Time Frame. Subsequent events within the Time Frame will trigger the Action.
Time Frame	The time period within which the Limit is enforced, specified in seconds.

In all cases except one, the Limit can be zero. If so, the specified Action will occur immediately when an incoming request matches the Event Criteria. For Include/Exclude criteria, the Limit must be set to at least one. If you wish to specify a Limit of zero, then the Origin Response Codes fields must be left empty. More about this in the Including/Excluding Requests section.

Event Criteria

Event criteria have two components:

• Key(s). These define the events that are being counted. See "Composing Keys," below.

• Include/exclude filters constrain the requests whose Keys are being counted. In other words, they define the segment of the incoming traffic stream that is being evaluated for possible events. See "Including/Excluding Requests," below.

Composing Keys

A Key consists of a field and a value. Within a Rate Limit Rule, they play a role like this:

"More than <LIMIT> requests with the same <KEY-VALUE> <KEY-FIELD> sent to <ASSIGNED-LOCATION> within <TIME-PERIOD> will cause <ACTION>."

Example: "More than three requests with the same username argument sent to the login form within one hour will cause the requestor to be banned for six hours."

Key Fields

A Key can be built upon any one of these four fields:

Field	Result
Header	All requests with the same value for the specified header will be counted together toward the Limit.
Cookie	All requests with the same value for the specified cookie will be counted together toward the Limit.
Argument	All requests with the same value for the specified argument will be counted together toward the Limit.
Attribute	All requests with the same value for the specified attribute will be counted together toward the Limit.

Multiple Keys

Multiple Keys can be defined within the same Rule. To create a new Key and open it for editing, select the "+" button next to the Key label.

If multiple Keys are defined, they are evaluated by combining them together with a logical AND. In other words, the cumulative count toward the Limit will be incremented whenever a request is seen that matches all of the Keys simultaneously. Different combinations of Keys will have separate Limit counts maintained for them.

Example. A Rule contains two Keys: "Attribute / Remote Address" and "Argument / Username". When the first request is received, an internal counter is created (set to a value of one) for this unique combination of Remote Address and Username. A second request is then received, originating from the same Remote Address and for the same Username; this causes the internal counter to be incremented up to two. A third request is then received from the same Remote Address but with a different Username; this causes a new internal counter to be created (and set to a value of one) for this combination.

The "Pair with" option: Changing the meaning of the Rate Limit

Below the list of Key(s), there is a checkbox labeled "Pair with." If this is checked, then a different type of Key can be added below it.

In the following discussion, "Key" will refer to the Key (or combination of Keys) defined above the "Pair with" checkbox.

"Paired Key" will refer to an optional, additional Key that is defined below the checkbox.

Adding a Paired Key changes the evaluation process. A Paired Key is not logically combined with the preceding Key; it is always evaluated separately.

Also, adding a Paired Key changes the meaning of the Rate Limit.

• If a Paired Key is not defined, an internal counter is maintained for each Key value, and incremented each time that value is encountered in a request.

• If a Paired Key is defined, an internal counter is maintained for each Key Value, and incremented each time a new, previously unobserved Paired Key value is encountered in a request.

Therefore, if a Paired Key is defined, the Rate Limit constrains the number of allowable Paired Key values for any given Key value.

So, the evaluation becomes something like this:

"More than <LIMIT> <PAIRED-KEY-VALUE> <PAIRED-KEY-FIELD>s per any one <KEY-VALUE><KEY-FIELD> sent to <ASSIGNED-LOCATION> within <TIME-PERIOD> will cause <ACTION>."

Note that the number of Key values is not being limited here. The limit is on the number of Paired Key values that each Key value is allowed.

Example: Let's say we want to allow an individual user to login from a maximum of two ASNs within one hour. (Perhaps the user is accessing our web application from a coffee shop's WiFi, and then a few minutes later, leaves the coffee shop and begins using the cell network instead.) We want to allow this possibility; however, if we receive requests from the same user originating from three or more ASNs within an hour, we want to treat this traffic more suspiciously. This is not possible merely by specifying two Keys, as described earlier in the "Multiple Keys" section. If we set up two Keys ("Argument / Username" and "Attribute / Organization") with a limit of 2, and assign it to our login form, then this will merely limit the number of times that the user can login from each ASN within an hour. Instead, we can set up one Key ("Argument / Username"), check the "Pair with" checkbox, and then set up the Paired Key ("Attribute / Organization"). Now the Limit will apply to the number of combinations of Username and Organization that are received for each specific Username.

Including/Excluding Requests

The Include and Exclude filters allow you to constrain the requests against which this Rate Limit Rule will be evaluated.

By default, an active Rate Limit Rule will be enforced upon all incoming requests. Specifying an Include and/or an Exclude filter will set limits to this enforcement.

1. Include and Exclude Tags

The Include filter will limit enforcement to the Tags specified in the Tags field under Include.

In the autofill field of the Include section, enter the name of an already existing Tag from Tag Rules. If more than one Tag is needed, click the "+" sign to the right of the field to add another Tag.

If there is a need to add a Tag which does not exist yet, create a new rule in Tag Rules. It will then be included in the list of Tags available for rate limiting.

All other requests in the traffic stream that do not have the Tags specified will not have the Rate Limit Rule enforced upon them.

Selecting Tags in the Exclude filter works in the identical manner as Include. Tags specified in the Exclude filter will exempt requests from enforcement that otherwise would have been evaluated. Exclude requests, which otherwise match the Include filter's parameters, will not have the Rate Limit Rule applied to them.

Note that the same Tags cannot be specified for Include as for Exclude.

2. Include and Exclude Origin Response Codes

For both Include and Exclude: In this field, specify the code(s) returned by the origin server which will be counted for this rule. This can be either a single three-digit code or a range of two three-digit codes separated by a hyphen, for example, 300-700.

Note that the same Origin Response Codes cannot be specified for Include as for Exclude.

If you wish the Limit to be set to zero, leave the Origin Response Codes field blank for both Include and Exclude.

Action

When a Rate Limit is triggered, the specified Action will occur.

There are various Actions available for selection here: 503 Service Unavailable, Challenge, Tag Only, and more. They are described in the documentation for the Action Response dropdown.

The Ban action is a unique option for Rate Limiting, and is described in detail below.

The Ban action

Most of the available Actions will not fully exclude an attacker that continues pressing the attack.

Example: Access to a login form is rate-limited to three requests per minute. An attacker tries to brute-force the login, and sends 60 requests per minute. The Rate Limit allows the first three requests, but then blocks the next 57 requests with a 503 error. However, after the minute has passed, the Rate Limit resets. The attacker is allowed another three attempts before being temporarily blocked again. This cycle can continue for as long as the attacker wishes. In effect, the Rate Limit is not preventing the attack; it is merely slowing it from 60 attempts per minute down to three attempts per minute.

The Ban action can be used to block (or take some other Action in response to) a Rate Limit violator for an extended period of time.

Example: As described above, a Rate Limit is created to allow three requests per minute, with an Action of 503 Service Unavailable. However, an additional Rate Limit rule is also defined: nine requests per three minutes, with an Action of Ban. The Ban has an Action of 503 Service Unavailable, and a duration of one hour. Reblaze allows multiple Rate Limits to be assigned to a single URL. Thus, both of the above rules can be assigned to the login form. Now an attacker tries to brute-force the login form, sending 60 requests per minute. The first three requests are allowed. The next six requests are blocked by the first Rate Limit. The tenth request triggers the second Rate Limit, and the Ban occurs. For the next hour, the attacker's requests will be blocked with a 503 error.

Second example: A hostile bot receives a bot challenge, which it fails. Reblaze will block the request. If the bot keeps re-submitting its request, it will continue to fail the challenges. However, each time the bot tries again, Reblaze has to issue a new challenge . To solve this problem, a second Rate Limit is created with a Ban action. Now a persistent bot will simply be Banned, saving the overhead of issuing continuous challenges.

Note that when setting up a Ban, the most common choices for its Action are to deny the violator's requests (via 503 Service Unavailable, Response, or Redirect). However, you can also choose Tag Only (to observe the violator's actions during the ban period), Challenge (to verify that the violating activity is not being done by bots), or Request Header (to mark the requests for further scrutiny by the backend).

Assigning Rate Limit Rules to URLs

In the main window, each Rule listing has a number for Linked URLs.

The number represents the number of URLs to which this Rule is currently assigned. URLs are linked after a new Rule is created, or they can be added later when editing an existing Rule.

URL linking is done at the bottom of the rule editing window:

When creating a new Rate Limit rule, the Links to URLs section does not appear until after the new rule is saved.

The Site dropdown contains a list of web applications that are currently defined in the Planet Overview.

After selecting a web application, a pulldown list will appear. It will contain the locations defined for this web application within the Security Profiles section of the Web Proxy page. To assign the current Rule to this location:

• Select a location from the pulldown

• Select the "+" button to add it to the list of assignments.

• Select the Save button at the upper right of the window.

More than one assignment can be performed at once. After adding the first assignment to the list, simply add additional ones as needed. When the list is complete, select the Save button.