Rate Limiter¶

Token-bucket rate limiter for controlling request throughput.

Enforces a global request rate across all concurrent browser pages. When multiple pages share a single RateLimiter, the combined throughput of all pages is capped at the configured limit.

Example:

# 15 requests per second = 900 per minute
limiter = RateLimiter(max_requests_per_minute=900)

# Before each request (called from multiple concurrent pages):
await limiter.acquire()   # blocks until a token is available
await page.goto(url)

# Disable rate limiting (default):
limiter = RateLimiter()   # or RateLimiter(max_requests_per_minute=None)

class intelliscraper.rate_limiter.RateLimiter(max_requests_per_minute=None)[source]¶

Bases: object

Token-bucket rate limiter shared across all concurrent pages.

The rate limit is enforced globally — if you have 4 concurrent pages and a limit of 900 requests/minute (15/sec), all 4 pages share that 15/sec budget, not 15/sec each.

Parameters:: max_requests_per_minute (int | None) – Maximum requests allowed per minute. Set to None or 0 to disable rate limiting (default).

enabled¶: Whether rate limiting is active.

max_rpm¶: The configured maximum requests per minute.

Example:

# No rate limiting (default)
limiter = RateLimiter()

# 15 requests per second across all pages
limiter = RateLimiter(max_requests_per_minute=900)

# Conservative limit for protected sites
limiter = RateLimiter(max_requests_per_minute=60)  # 1/sec

property enabled: bool¶: Whether rate limiting is active.

property max_rpm: int¶: The configured maximum requests per minute.

async acquire()[source]¶

Wait until a request token is available.

If rate limiting is disabled, returns immediately. Otherwise, blocks until enough time has elapsed since the last request to stay within the configured rate.

This method is safe to call from multiple concurrent tasks — an asyncio.Lock ensures only one task checks and updates the timestamp at a time.

Return type:: None