How Go's rate.NewLimiter Works: Why burst Doesn't Cap ConcurrencyArticle · digest for October 31
Infographic diagram of Go's token bucket algorithm showing an overflowing burst reservoir and a concurrency gate.
Article3 sections≈ 5 min

How Go's rate.NewLimiter Works: Why burst Doesn't Cap Concurrency

Every backend engineer eventually runs into the problem of protecting a service from overload. Traffic spikes, downstream integrations start flaking, and the database groans under a barrage of queries. For a Go developer, the first instinct is to grab the battle-tested golang.org/x/time/rate package and slap a limiter onto incoming HTTP endpoints.

And right there lies one of the most common—and sneakiest—architectural illusions in Go. You write limiter := rate.NewLimiter(100, 10), thinking, "Great, now at most 10 requests can run at once, and the database is safe from storms." A week later, production catches fire anyway: your PostgreSQL connection pool is exhausted, p99 latency shoots through the roof, and the service stops responding.

Let's pop the hood on rate.Limiter and inspect how the gears actually turn.

Token Bucket Math: Rate vs. Concurrency

golang.org/x/time/rate is built on the classic Token Bucket algorithm. The mechanics are dead simple:

  1. The bucket holds at most $b$ tokens—that's the burst parameter.
  2. New tokens drip into the bucket at a steady rate $r$—that's limit (tokens per second).
  3. Every incoming request must grab a token from the bucket. If one is available, the request proceeds immediately. If the bucket is empty, the request is either rejected on the spot (Allow()) or blocks until a token refills (Wait(ctx)).

Notice the fundamental distinction: A token bucket controls event frequency over time (rate limit), and it knows nothing at all about how many tasks are currently running (concurrency limit).

Picture this: the bucket is full with 10 tokens (burst = 10). Ten parallel requests slam into your service at the exact same millisecond. The limiter dutifully hands all 10 requests a green light. All 10 goroutines storm the database simultaneously. Now, if each of those requests runs a heavy SELECT that takes five seconds, your database spends those full five seconds absorbing 10 concurrent sessions! And if the bucket refills at 100 tokens per second, the very next second brings another wave straight into the DB.

When You Need rate.Limiter vs. a Semaphore

To protect downstream dependencies—databases or external APIs with hard limits on open sockets—rate.Limiter alone won't cut it. You want two independent mechanisms working together:

  • rate.Limiter: smooths out spikes in request frequency over time.
  • A weighted semaphore (semaphore.Weighted from golang.org/x/sync/semaphore): puts a hard ceiling on how many workers execute in parallel.

Here's the clean engineering pattern for combining both tools:

package main

import (
    "context"
    "fmt"
    "net/http"
    "time"

    "golang.org/x/sync/semaphore"
    "golang.org/x/time/rate"
)

type ProtectedService struct {
    rateLimiter *rate.Limiter
    concurrency *semaphore.Weighted
}

func NewProtectedService(rps float64, burst int, maxConcurrent int64) *ProtectedService {
    return &ProtectedService{
        rateLimiter: rate.NewLimiter(rate.Limit(rps), burst),
        concurrency: semaphore.NewWeighted(maxConcurrent),
    }
}

func (s *ProtectedService) HandleTask(ctx context.Context, taskID int) error {
    // 1. Check the rate limit: wait for a token over time
    if err := s.rateLimiter.Wait(ctx); err != nil {
        return fmt.Errorf("превышен таймаут ожидания токена: %w", err)
    }

    // 2. Cap concurrency: acquire a slot in the semaphore
    if !s.concurrency.TryAcquire(1) {
        return fmt.Errorf("все слоты параллельной обработки заняты")
    }
    defer s.concurrency.Release(1)

    // Simulate useful work (a database query)
    time.Sleep(200 * time.Millisecond)
    return nil
}

Look closely at TryAcquire(1): if all parallel slots are already tied up by slow operations, the service fails fast and returns an error (HTTP 429 or 503) instead of piling up stalled goroutines in memory. If you remember how Go concurrency behaves under pressure, managing goroutine lifetimes with contexts and semaphores is the bedrock of service reliability.

The Nuances of Allow, Wait, and Reserve

The rate package gives you three primary ways to interact with the bucket, each with a distinct job:

// 1. Allow(): non-blocking check (ideal for API gateways)
if !limiter.Allow() {
    http.Error(w, "Too Many Requests", http.StatusTooManyRequests)
    return
}

// 2. Wait(ctx): blocking wait with context cancellation support
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
if err := limiter.Wait(ctx); err != nil {
    // Context canceled or timed out before a token became available
    return err
}

// 3. Reserve(): reserve a token ahead of time and get the wait duration (Reservation)
r := limiter.Reserve()
if !r.OK() {
    return fmt.Errorf("запрос превышает максимальный burst")
}
time.Sleep(r.Delay()) // Sleep for the exact delay until the token is ready

One subtle gotcha: if you call Wait(ctx) asking for more tokens than the burst capacity (limiter.WaitN(ctx, n) where n > burst), it fails immediately—a bucket physically can't hold more tokens than its max size.

Think of burst as a shock absorber for traffic spikes, not a circuit breaker for parallel load. Keep those two concepts separate in your code, and your service won't buckle the next time a sudden wave of requests hits.

Want a deep diveVotes help us choose what to break down in detail. This material has 0 votes.
Sources · 2