Usage Controls

Self-Hosting Documentation Access

This section requires a password to access. Interested in self-hosting? Contact sales to learn more.

Password:

Access Documentation

Self-Hosting Documentation Access Granted Logout

A self-hosted deployment runs on infrastructure you provision and pay for, so unbounded usage shows up on your bill and in your cluster. LlamaCloud gives you two independent controls over that:

• API rate limits bound how fast a project may call the API. They protect the deployment from request floods, and reject with HTTP 429.
• Usage caps bound how many credits an organization or project may consume in a day or a month. They protect your budget, and reject with HTTP 402.

Rate limits ship with generous defaults. Usage caps ship off. Neither is a substitute for the other: a rate limit throttles request frequency but permits unlimited total spend over time, while a usage cap allows any request rate until the credit budget runs out.

Usage caps

A usage cap is a credit budget for a rolling window. When consumption in the window reaches the cap, further job-creating requests are rejected with 402 and the message This <scope> has reached its <window> usage limit. Read operations are unaffected — a capped organization can still retrieve results for work it already submitted.

Caps apply to job creation across Parse, Extract, Classify, Split, and LlamaSheets.

Turning caps on

Caps have three modes, set with config.usageCaps.enforcementMode:

Mode	Behavior
off (default)	No evaluation at all. Costs nothing on the request path.
observe	Evaluate every request and log what would have been blocked, but allow it.
enforce	Reject a request that breaches a cap.

config:
  usageCaps:
    enforcementMode: observe
    defaultDailyCredits: 5000
    defaultMonthlyCredits: 100000

defaultDailyCredits and defaultMonthlyCredits are the deployment-wide default caps applied to every organization that has no override of its own. Leave them unset and organizations are uncapped, which makes enforce inert — the mode is the switch, the credit values are the limit.

Caution

0 is a cap, not an absent one. It means zero credits, so in enforce it blocks job creation for every organization it applies to. To leave organizations uncapped, omit the key rather than setting it to 0.

Roll out a cap safely

A cap that lands at the wrong number reaches your users as a 402. Get to enforce in this order:

1. Set the credit values. defaultDailyCredits and defaultMonthlyCredits with enforcementMode: observe. The mode alone enforces nothing; a mode without values evaluates nothing.
2. Run in observe. Every request is evaluated and allowed, so a cap that is far too low costs you a log line instead of an outage.
3. Watch what would have been blocked. In observe, a breach still counts as blocked on llamacloud_usage_cap_evaluations_total, labelled mode="observe". Those are the requests enforce would have rejected, and the organizations behind them are the ones to talk to first.
4. Size the cap against a full billing month. Month-end and quarter-end are when real usage peaks; a cap fitted to a quiet fortnight rejects legitimate work later.
5. Switch to enforce, and alert on failed_open. Enforcement that degrades stops blocking silently — see Caps fail open.

Organization caps and project caps

Caps are evaluated at two scopes, and a breach of either blocks the request:

• Organization: total credits across all of the organization’s projects.
• Project: credits for one project on its own.

The tightest breached cap is the one reported, checked in the order project daily, project monthly, organization daily, organization monthly.

Caution

A project usage cap never inherits — not from its organization, not from the deployment-wide default.

An organization cap bounds the total across all of its projects. Applying that same number to each project individually is a much larger budget, not the same one.

Setting defaultDailyCredits caps every organization and caps no project.

Caps fail open

If a cap cannot be resolved, or the usage read fails or times out, the request is allowed. A degraded cap check never blocks legitimate work — but it also stops enforcing, silently. Watch the failed_open outcome to catch that:

Metric	Labels	Meaning
llamacloud_usage_cap_evaluations_total	outcome, mode	Cap evaluations. outcome is one of allowed, blocked, failed_open. A rising failed_open means enforcement is degraded, not that nothing is being blocked.

A sustained non-zero failed_open rate in enforce mode should be treated as an outage of the control, not as quiet success.

A cap check also reads a periodically aggregated usage total rather than counting each job as it lands, so it can trail the most recent work and allow a request that a perfectly current count would block. For the same reason an organization’s first capped check can read zero, typically for under a minute after its usage starts. Enforcement catches up on its own once the total lands.

API rate limits

Rate limits are per-project defaults, expressed as either queries per second or a request count over a window. Set them under config.rateLimits in your values.yaml:

config:
  rateLimits:
    parseApiCreationQueries: 500
    parseApiCreationSeconds: 10
    parseApiQueryQps: 80
    parseApiListQps: 5

Value	Default	Limits
parseApiCreationQueries / parseApiCreationSeconds	500 per 10s	Parse job creation
parseApiQueryQps	80	Parse job reads
parseApiListQps	5	Parse job list calls
extractApiCreationQps	40	Extract job creation
extractApiQueryQps	80	Extract job reads
extractApiListQps	5	Extract job list calls
extractAgentCreationQps	10	Extract agent creation
classifyApiCreationQps	40	Classify job creation
classifyApiQueryQps	80	Classify job reads
classifyApiListQps	5	Classify job list calls
splitApiCreationQps	40	Split job creation
splitApiQueryQps	80	Split job reads
spreadsheetCreationQpm	60	LlamaSheets job creation (per minute)
spreadsheetApiQueryQps	80	LlamaSheets job reads
spreadsheetApiListQps	5	LlamaSheets job list calls
usageApiQueryQps	20	Usage metrics API

List operations carry the lowest defaults deliberately: they are the most expensive per call and the easiest to issue in a loop.

Overriding a limit for one organization

Deployment-wide defaults are a floor for the whole cluster; a single organization usually needs its own number. Global administrators set overrides from Settings → Admin → Quota Management, searching for the organization by name or by id.

The page has two views:

• Overrides — only the values someone set deliberately, for this organization and for any of its projects. These are the rows you can change or remove.
• All Limits — every limit the organization currently resolves and where each value came from: its own override, its plan tier, or the deployment-wide default. Use it to check an effective number before changing it. Plan tier and deployment-wide values cannot be deleted here.

Daily Usage Credits Limit and Monthly Usage Credits Limit override the deployment-wide cap for one organization, exactly as the rate limit entries override theirs. Use an override to exempt a high-volume organization from a deployment-wide cap, or to hold one noisy organization below it.

Capping a single project

Adding a daily or monthly usage cap offers an Applies to choice: the organization, or one of its projects. Organization-wide bounds the combined total across every project; choosing a project bounds that project alone.

Only the usage caps offer this. Every other limit inherits, so an organization override already governs its projects and a project-level copy would say nothing new. And because caps do not inherit, capping a project is the only way to bound it — a project with no cap of its own stays uncapped no matter what its organization is set to.

Setting up global administrators is covered in Global Admin Setup.

How a limit is resolved

Every limit is a quota, and quotas resolve from most to least specific:

organization override → plan tier default → deployment-wide default

The first of those that exists wins. Where a limit can be set, and what it falls back to, differs by scope:

Scope	Set from	Falls back to	Result when nothing is set
Organization	Admin UI override, then plan tier, then Helm values	The next level down the chain	The deployment-wide default
Project (rate limits)	Its organization, then plan tier, then Helm values	The next level down the chain	The deployment-wide default
Project (usage caps)	A cap set on that project in the admin UI	Nothing	Uncapped

Deployment-wide defaults come from your Helm values, and your deployment stores them on startup. Changing a value takes effect after the backend restarts, not on the next request.

Organization overrides are set from the admin UI and take effect immediately.

Caution

Clearing a value you previously set does not remove the default it wrote. The startup sync writes and updates defaults; it never deletes them. To lift a limit you already applied, set it to a higher value rather than removing the key.

That startup sync is also what carries the Helm values into the deployment. If yours narrows the sync to a subset of quota types, the two usage cap types have to be in that subset — otherwise defaultDailyCredits and defaultMonthlyCredits are silently ignored.

Check what an organization consumes

Caps are only as good as your picture of current consumption. Settings → Admin → Global Usage reports credit usage per organization and project, and the usage metrics API exposes the same aggregation for export into your own reporting. It is where step 4 of Roll out a cap safely gets its numbers.

Related configuration

• Autoscaling Configuration — sizing the deployment to the load the limits above let through.
• LlamaParse Throughput — worker and model concurrency, which bound how fast accepted work is processed.
• Global Admin Setup — granting the access the Quota Management page requires.