Rate Limits and Backpressure: Spending an RPC Budget

An engine's request budget is spent mostly on confirmation, not on submission. Every in-flight transaction is polled repeatedly until it resolves, so requests scale with attempts multiplied by their lifetime rather than with the number of swaps. Backpressure is the mechanism that turns that pressure into a deliberate slowdown instead of a wall of refusals.

The reason this deserves its own note is that rate limiting is usually the first ceiling an engine meets, and it is almost always met in the wrong place. Operators size their endpoint plan from the number of swaps they intend to make, then discover that the confirmation path alone consumes several times that figure, and the resulting failure looks like a network problem rather than a capacity planning miss.

Where the request budget actually goes

Read the frequency column and the shape of the problem is obvious. Submission is a small share of requests. Confirmation dominates because it repeats, and it repeats for longer exactly when the network is slow, which is when your endpoint is also under pressure from everyone else. The load profile is worst precisely when capacity is scarcest.

The two cheapest structural improvements are batching confirmation queries by signature where the endpoint supports it, and lengthening the poll interval as an attempt ages. A transaction two seconds old is worth checking often; the same transaction at forty seconds is not going to resolve any faster for being asked more frequently.

The shapes a rate limit can take

Limits are not one thing, and a design that assumes requests per second will be surprised by the others. Four shapes appear in practice, often simultaneously.

A simple request rate caps calls in a window. A weighted or credit-based limit assigns different costs to different methods, so a heavy call can consume many times what a light one does, and a budget expressed in raw request counts is meaningless. A concurrency limit caps simultaneous in-flight requests regardless of rate, which is what a design with unbounded parallelism runs into first. And a data or payload limit caps how much you can pull, which is what large account scans meet.

Because the shapes coexist, the only reliable approach is to instrument your own usage rather than to model the provider's policy. Count requests by method, record refusals by method, and record the concurrency you actually reached. If the numbers you keep match the units the limit is expressed in, you can plan; if they do not, you are guessing about somebody else's accounting.

Backpressure is not retry

These get conflated constantly and they operate at different levels. Retry with backoff is a per-call decision about a call that already failed. Backpressure is a system-level decision about whether to admit new work at all. An engine can have flawless backoff and no backpressure, and it will behave badly in exactly the way that matters.

The failure mode is easy to picture. Confirmations slow down, so attempts stay in flight longer, so the confirmation poller makes more requests, so the endpoint starts refusing, so those requests retry with backoff. Meanwhile the scheduler, which knows none of this, keeps releasing new intents on its original pace. In-flight count climbs, request volume climbs with it, and the engine converges on a state where almost every request is a retry of a refusal.

Backpressure breaks that loop by connecting the sender's view back to the scheduler. The signal should be a level rather than a count, because the interpretation belongs where the context is, and the scheduler should respond by pacing rather than by stopping, so the system degrades along a curve instead of a cliff.

Queue depth as the control signal

Depth is the number of units of work in flight: released, not yet settled. It is the right control signal for three reasons. It rises before errors appear, so it gives the scheduler warning. It is a single number with a natural comparison point, the concurrency ceiling. And it is directly actionable, since the only lever that changes it is the release rate.

Error rate, by contrast, is a lagging indicator that also mixes causes. A rise in refusals could be a rate limit, an endpoint incident, or a construction bug producing malformed transactions. Depth does not care why; it says the system is not keeping up, which is the thing the scheduler needs to know.

depth      = released - settled
utilisation= depth / ceiling

utilisation < 0.6   -> ok
0.6 .. 0.85         -> elevated : pace * 0.5, shed optional work
> 0.85              -> critical : hold releases, keep confirming

also elevate if:
  p90 settle time > 3x baseline
  refusal rate > threshold over a rolling window

The thresholds above are a starting shape rather than a recommendation, because the right values depend on your ceiling and your latency distribution. What matters is the structure: a bounded ratio, a graded response, and a critical state that stops new work without touching the confirmation path.

A worked capacity example

The following arithmetic is illustrative and uses inputs you would measure yourself. Its purpose is to show that the binding constraint is usually discoverable on paper before it is discoverable in production.

Little's law says the average number of items in a system equals the arrival rate multiplied by the average time each spends there. Suppose an engine releases one swap every 36 seconds, so the arrival rate is roughly 0.028 per second, and suppose a swap settles in 8 seconds on average. Average in-flight work is 0.028 multiplied by 8, which is about 0.22. One account is comfortably enough and depth is never a concern.

Now take a congestion window where settle time rises to 45 seconds. In-flight becomes 0.028 multiplied by 45, or about 1.25. Still small, but the request profile has changed dramatically: each of those in-flight items is being polled for five times as long. If the poller checks every two seconds, per-attempt confirmation requests rise from about four to about twenty-two.

1. Count requests per swap at baseline: one quote, one submission, four confirmation polls, one sampled simulation every ten swaps. Call it seven.
2. Count them under congestion: one quote, perhaps two submissions after an expiry and rebuild, twenty-two confirmation polls. Call it twenty-five.
3. Multiply by the release rate. At 100 swaps an hour, baseline is 700 requests an hour and congestion is 2,500, against an unchanged release plan.
4. Compare against the endpoint allowance you are actually paying for. If the congestion figure exceeds it, the run will fail during congestion, which is the only time it matters.
5. Now apply the fix that costs nothing: an ageing poll interval. Polling every two seconds for the first ten seconds and every ten seconds afterwards cuts the congestion figure by more than half without changing the outcome of a single swap.

The general lesson is that capacity planning for an engine has to be done against the bad case, because the good case never generates enough load to be interesting. Planning against the average is how an engine discovers its ceiling on the worst afternoon of the month, and it applies just as much to a self-built system as to a hosted multi-DEX Solana volume bot, where the same request profile exists behind somebody else's capacity plan.

Shedding load on purpose

Load shedding means choosing what not to do when you cannot do everything. The alternative is not doing everything anyway and letting the endpoint choose for you, which it will do arbitrarily and at the worst possible moment.

The shed order should be written down before it is needed, because during an incident nobody derives it correctly from first principles.

• Sampled simulation. Pure verification, valuable but not on the critical path for work already in flight.
• Balance and account refreshes. Defer them, mark the cached values as stale, and let reconciliation catch up afterwards.
• Cosmetic and dashboard queries. Anything whose only consumer is a screen can wait.
• New releases. This is the big lever and it belongs above confirmation in the order, not below it.
• Route quote refreshes for intents not yet released. Holding a stale quote is fine if the intent has not gone anywhere.
• Confirmation polling. Last, and only by lengthening intervals, never by stopping. Stopping converts pending into unknown.

The ordering principle is that you shed work whose loss costs information before work whose loss costs certainty. A skipped simulation costs you a check you can repeat. A skipped confirmation costs you the ability to say what happened, and that is unrecoverable.

Multiple endpoints and their traps

Spreading load across several endpoints raises your effective ceiling and introduces a consistency problem. Endpoints can be at different slots, which means a transaction accepted by one may genuinely not be visible from another for a short period. An engine that submits to endpoint A and confirms against endpoint B will manufacture unknowns that have nothing to do with the network.

Two rules make multi-endpoint designs behave. Pin confirmation for a signature to the endpoint that accepted it, at least for the first several checks, falling back to others only after the transaction is old enough that slot skew cannot explain absence. And keep per-endpoint accounting, because a shared credit counter across endpoints with different policies is a number that describes nothing.

The other trap is that failover hides degradation. If endpoint A starts refusing and the engine silently moves to B, the run looks healthy while your redundancy quietly disappears. Failover events belong in the metrics with the same prominence as errors, because a system running on its last endpoint is one incident from stopping and nobody has been told.

The circuit breaker

A circuit breaker stops calling something that is clearly not working, waits, then tries a single probe before restoring normal traffic. It is worth having in the sender because it turns a sustained failure into a bounded one, and because it makes the failure visible as a state rather than as a stream of errors.

closed  : normal traffic, count failures in a rolling window
  failures > threshold -> open

open    : reject immediately, do not call, start a cooldown timer
  cooldown elapsed -> half_open

half_open : allow exactly one probe request
  probe succeeds -> closed, reset counters
  probe fails    -> open, cooldown doubles up to a cap

The rule that keeps a breaker useful is that opening it must be visible. A breaker that opens silently is an engine that has stopped working while reporting nothing, which is worse than an engine that is failing loudly. Emit a state change event, surface it to the operator, and record the duration, because breaker-open time is one of the more honest measures of how a run really went.

Backpressure review checklist

• Is request volume measured per method, or only inferred from swap counts?
• Does the confirmation poll interval lengthen as an attempt ages?
• Is there a pressure level flowing from the sender back to the scheduler?
• Is queue depth against the ceiling the primary control signal, rather than error rate?
• Has capacity been calculated against the congestion case rather than the average?
• Is the shed order written down, with confirmation polling last?
• Is confirmation for a signature pinned to the endpoint that accepted it?
• Are failover and circuit breaker state changes surfaced as events rather than logged quietly?

An engine that answers yes to all eight will slow down under pressure and finish, which is the only behaviour worth engineering for. Read the scheduler note for the upstream half of the handshake, and the observability note for how depth, refusals and breaker state become a picture an operator can act on.

Questions this note gets asked