AI Security
LLM API Security:
Rate Limiting, Auth, and Prompt Leakage
Published: 2026-05-17 22:59 PDT (Oregon)
Exposing an LLM via API introduces security concerns distinct from traditional web APIs. System prompt leakage through carefully crafted queries, cost exhaustion attacks that burn through API budgets, and the challenge of rate limiting non-deterministic token-generating endpoints all require purpose-built approaches that standard API security doesn't cover.
Where LLM API Security Differs
Traditional API security concerns — authentication, authorization, input validation, rate limiting — all apply to LLM APIs. But LLM APIs introduce threat categories that have no traditional analog. The API endpoint isn't just processing requests — it's executing a model that can be manipulated through the content of those requests to behave in ways the API designer didn't intend.
The key difference is that the input is processed by the model's weights, not just your validation logic. An input that passes all your parameter validation may still cause the model to output your system prompt, generate harmful content, or make downstream tool calls that weren't part of the intended interaction flow.
LLM-Specific API Security Concerns
System prompt leakage
System prompts contain business logic, personas, safety instructions, and sometimes credentials. Users who can query your API can systematically probe for system prompt contents using completion attacks, translation tricks, and indirect extraction. The system prompt should be designed as if it's public — because it effectively is.
Cost exhaustion attacks
LLM inference is priced by token. An attacker who submits requests designed to maximize output token count — "write an extremely detailed 10,000-word essay on..." — can exhaust per-user budgets or run up significant infrastructure costs. Token-based rate limiting is fundamentally different from request-count rate limiting.
Cross-user prompt injection
In multi-tenant LLM applications, one user's input that manipulates the model's context can affect the model's responses to subsequent queries — especially in applications that share conversation history or use retrieved context from a shared vector store. One malicious user can contaminate the context for others.
Session and conversation isolation
Conversation histories, RAG retrieved contexts, and tool call results must be strictly isolated between users and sessions. Failure to isolate means a user may receive content from another user's conversation — a data leakage violation that doesn't require any active attack.
Authentication and Authorization
LLM API authentication should treat API keys as secrets — not as identifiers. Keys embedded in client-side JavaScript, mobile app binaries, or frontend code will be extracted and abused. Server-side proxying, where the client calls your backend and your backend calls the LLM provider, keeps API keys out of client contexts entirely.
Developer builds a React app that calls the OpenAI API directly from the browser. The API key is in the client JavaScript bundle. Any visitor can extract it from DevTools, use it for unlimited API calls on the developer's account, or sell it. GitHub secret scanning regularly discovers thousands of valid API keys per month in public repositories — many from developers who didn't realize they'd committed them.
Authorization for LLM endpoints requires defining what each authenticated user is permitted to request. Not just which endpoints they can call, but what capabilities the model is permitted to exercise on their behalf — which tools, which data sources, with what output constraints. User-level capability grants, not just endpoint-level authentication.
Rate Limiting for LLM Endpoints
Request-count rate limiting is insufficient for LLM endpoints because cost is proportional to tokens, not requests. A 1 request/second limit means nothing if each request generates 10,000 tokens. Effective rate limiting for LLM APIs requires token budgets: per-user, per-minute input token limits and output token limits, enforced at the proxy layer before requests reach the model.
Streaming responses add complexity — the full output token count isn't known until the response completes. Implement both pre-request limits (maximum context length) and post-request accounting (token consumption tracked against the user's budget), with circuit breakers that terminate unexpectedly long responses.
Mitigations
- Always proxy LLM calls through your backend. Client applications should never hold LLM provider API keys. All model calls go through a backend service that handles authentication, authorization, rate limiting, and logging before forwarding to the provider.
- Implement token budget rate limiting. Track input and output token consumption per user, per time window. Enforce hard limits that prevent individual users from generating costs above defined thresholds. Alert on anomalous consumption patterns that may indicate automated abuse.
- Log every request and response for audit. Store conversation histories server-side with user attribution. This enables incident investigation, anomalous pattern detection, and compliance with data retention requirements. Ensure logging infrastructure is isolated from the model serving path.
- Design system prompts as public documents. Never embed secrets, credentials, or business logic in system prompts that would be damaging if exposed. Assume every system prompt will eventually be extracted — because under sufficient adversarial probing, it will be.
- Enforce strict conversation isolation in multi-tenant deployments. Each user's conversation context, retrieved documents, and tool call results must be stored and retrieved in strict isolation. Cross-contamination of context between users is both a security and a privacy violation.