Based on the runtime statistics collected in previous slots, FlowSlot will use pre-set rules to decide whether the incoming requests should be controlled.
SphU.entry(resourceName) will throw
FlowException if any rule is triggered. You can customize your own logic by catching this exception.
One resource can have multiple flow control rules. FlowSlot traverses these rules until one of them is triggered or all rules have been passed.
A flow control rule consists of the following elements, and you can use a combination of different elements to cater for different flow control needs.
You can control either by thread count or QPS. It is defined by the
grade field in
FlowRule. Both concurrent thread count and request count are collected at runtime, and you can run the following command to view the statistics:
The output is as follows:
idx id thread pass blocked success total aRt 1m-pass 1m-block 1m-all exception
2 abc647 0 460 46 46 1 27 630 276 897 0
This mode is usually used to protect threads from being occupied. If a resource takes a long time to finish, threads blocked in this resource will begin to be occupied. The longer the response takes, the more threads might be occupied.
Besides thread count, there are 2 other ways to achieve this, thread pool or semaphore.
The benefit of using thread pools is that you can isolate business logic completely by pre-allocating thread pools. But it also brings us extra costs of context switch and additional threads. If the incoming request is already handled in a separated thread, for instance, a servlet request, it will almost double the thread count if using the thread pool mode.
So we recommend to use concurrent thread count flow control, which represents lightweight semaphore isolation.
When QPS exceeds the threshold, we will take actions to control the incoming requests, and this can be done by configuring the
controlBehavior field in
1. Immediately reject (
This is the default behavior. The exceeded requests are rejected immediately and the FlowException is thrown.
If the usage of the system has been low for a while, but all of a sudden a large amount of requests comes in, the system might not be able to handle all these requests at once. However, if we steadily increase the incoming requests and allows the system to warm up, it may be able to handle all the requests eventually.
This warm-up period can be configured by setting the
warmUpPeriodSec field in
3.Rate limiter (
This strategy strictly controls the interval between requests. In other words, it allows requests to pass at a uniform rate.
This strategy is an implement of leaky bucket. It is used to handle requests at a stable rate and is often used to process burst requests instead of rejecting them. For instance, a sudden inflow of messages. When a large number of requests arrive at the same time, the system can handle all these incoming requests at a fixed rate.
We use the NodeSelectorSlot to establish the paths of resources, and the ClusterNodeBuilderSlot to collect the caller's runtime data.
ContextUtil.enter(resourceName, origin), the parameter
origin indicates the identity of the caller.
ClusetNodeBuilderSlot will collect this info, and uses it to perform flow control.
This information can be displayed by the following command:
idx origin threadNum passedQps blockedQps totalQps aRt 1m-passed 1m-blocked 1m-total
1 caller1 0 0 0 0 0 0 0 0
2 caller2 0 0 0 0 0 0 0 0
origin can be defined by the field
limitApp in FlowRule. This field has the following values:
default: No specific caller. If the total value of this resource exceeds the threshold defined in this rule, the incoming request will be blocked.
<<origin>>: A specific caller has exceeded the threshold defined in the rule.
other: This rule applies to requests from a caller that is not defined explicitly in the
origin field for this resource.
The path is maintained in
NodeSelectorSlot. For example, the resource nodes can come from either entrance1 or entrance2.
We can shape the flow by setting the field
ref_identity to a specified entrance.
For instance, two resources will access the same database record. ResourceA will read records from the database, resourceB will write records to the database. The frequency of ResourceA accessing the database depends on ResourceB. We can achieve this by configuring a rule for ResourceA with the value of the
strategy field as
RuleConstant.RELATE, and the value of
ref_identity as ResourceB.