Skip to contents

Create or check a rate guard

Source: R/rate_guard.R
rate_guard.Rd

Rate guards are explicit stateful environments used to cap token and request budgets for LLM workflows. Unbounded consumption is OWASP LLM06:2026; see https://github.com/GenAI-Security-Project/GenAI-LLM-Top10/tree/main/2026/final.

Usage

rate_guard(
  max_tokens = NULL,
  max_requests = NULL,
  max_output_tokens = NULL,
  max_tool_calls = NULL,
  max_elapsed_seconds = NULL,
  window_seconds = 3600L,
  strict = FALSE,
  concurrent = FALSE,
  backend = NULL,
  show_stats = FALSE
)

Arguments

max_tokens

Maximum tokens per window, NULL, or an existing shieldr_rate_guard when checking a guard with rate_guard(guard).

max_requests

Maximum requests per window, or NULL.

max_output_tokens

Maximum output tokens reserved before each model call and enforced on the returned output estimate.

max_tool_calls

Maximum tool requests in one guarded chat.

max_elapsed_seconds

Optional model-call wall-time limit.

window_seconds

Window length in seconds.

strict

Whether secure_chat() should reserve estimated prompt tokens before calling the model.

concurrent

Whether to persist counters in a locked state file shared by worker processes on the same machine. Requires filelock.

backend

Optional shared quota backend as a list of usage, reserve, and rollback functions. This lets server deployments provide Redis, database, or service-backed atomic accounting without a core dependency.

show_stats

Show construction or check time and available usage metrics as messages.

Value

When creating a guard, a shieldr_rate_guard environment. When checking a guard, TRUE if usage is within limits.

Details

Calling rate_guard() with limits creates a new shieldr_rate_guard environment. The environment stores counters for the current window and exposes two methods:

  • $usage(): returns current counters and configured limits.

  • $reserve(tokens, requests): atomically checks projected usage and then increments counters when the reservation stays within limits.

  • $update(tokens, requests): backward-compatible alias for $reserve().

  • $rollback(tokens, requests): subtracts a previous reservation after a guarded operation fails before completion.

Calling rate_guard(guard) checks an existing environment and returns TRUE if all counters are within limits. Reservation methods fail before projected usage exceeds the configured token or request limit. Limits set to NULL are disabled for that dimension.

Windows reset automatically when window_seconds has elapsed. This object is intentionally stateful; it is the one place where llmshieldr expects mutable state, because rate limiting is inherently session-based.

Concurrency

The rate guard is not safe for concurrent use by default. Parallel or async R code (future, parallel, callr) that shares a single guard environment will produce inaccurate counts. Use concurrent = TRUE and install the filelock package to persist counters in a locked state file shared by serialized or forked copies of the guard on one machine. Cross-machine coordination requires backend=.

Pre-call Reservation

With strict = TRUE, secure_chat() reserves an estimated prompt token cost and one request before the model call, then records only the positive difference between the actual token estimate and the reserved amount after the call. If the chat call or output scan fails, the pre-call reservation is rolled back. This makes shared guards more useful under bursty load, but estimated tokens may differ from actual usage. Strict mode is recommended when multiple callers share one guard.

Examples

guard <- rate_guard(max_tokens = 100)
guard$reserve(tokens = 10)
rate_guard(guard)
#> [1] TRUE