Repository navigation
docs: gateway rate and concurrency caps are configurable on self-hosted - #109
Merged
Merged
Conversation
M201/M202/M203 and the API rate-limits section said the caps were fixed with no setting to change them. Since llm-gateway #2794 and #2724 they read MANIFEST_CONCURRENCY_MAX, MANIFEST_RATE_MAX_REQUESTS and MANIFEST_IP_RATE_MAX_REQUESTS. Document the variables, keep Cloud as fixed, and note the counts are per gateway instance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
M201, M202, M203 and the API rate-limits section still say the gateway caps are fixed with no setting to change them. That was true when it was audited (decisions.md, 2026-08-14 / 08-16), but the gateway changed since: llm-gateway #2794 made the concurrency cap configurable and #2724 did the same for the two per-minute caps.
MANIFEST_RATE_MAX_REQUESTS,MANIFEST_IP_RATE_MAX_REQUESTS,MANIFEST_CONCURRENCY_MAXin.envTHROTTLE_LIMITrow no longer calls the gateway caps fixedRoot pages and their
llm-gateway/docs/twins edited together.node scripts/check-coherence.mjsis GREEN.Not changed: M201 is titled "Per-user rate limit" but the gateway counts per workspace.