Repository navigation
docs(integration-toolkit): large data loads, the 503 for exhausted capacity, and sandbox organizations - #190
Merged
Conversation
added 4 commits
November 13, 2026 15:47
…pacity, and sandbox organizations Adds a Large Data Loads page: how everyday traffic and large loads are processed, what a large load looks like from outside, the 503 response with Retry-After and BULK_CAPACITY_EXHAUSTED (whole and mixed requests), file imports that wait for capacity, and planning a migration. Adds the Sandbox Organizations page at the address the Integration Hub banner and the SANDBOX_LANE notice link to, for the new sandbox behaviour. Adds the 503 where retries are described (inbound getting started, interactive inbound closing rules and rate limits, replay) and cross-links from the overview and the configuration page.
… import Quotes the reworded 503 messages: the one for a request refused whole, the top-level message of a mixed request and the message on each refused event, and the new message of an import that gave up waiting. Shows a file import that waits for capacity through progress.waiting_for_capacity (since, retry_at, gives_up_at), with an example job, and drops the sentence that its progress simply does not advance.
…rged behaviour - 503 INGEST_THROTTLED (Retry-After: 2) beside 503 BULK_CAPACITY_EXHAUSTED (Retry-After: 60) - size limits: 422 EVENT_TOO_WIDE above 5,000 updates per event, 422 REQUEST_TOO_WIDE above 25,000 per request, observed only in staging and production for now - order of events across the start and the end of a large load, and the time limit that is the exception - events that produce very many updates, managed calls, file imports waiting every 60 seconds for up to 6 hours - replays: 503 when nothing was queued, 200 with Retry-After when throttled part of the way - the processing is switched off in every environment: the pages say so
- New section Bulk Import Windows: booking a window in the Integration Hub (Monitoring tab) or with GET/PUT/DELETE /v1/bulk-imports/window, the rules, the lane status, the permissions. Planning a Migration now says to book one. - Large loads have been processed this way in production since 2026-11-19: the "Not active yet" notes are gone, and so is the note that production only observes the size limits (they are enforced). - When a load ends: after 15 minutes without events, or after 3 minutes when only a trickle follows a load that ran for at least 30 minutes. - Sandboxes: a window works there too, at the sandbox rate.
karlrank
marked this pull request as ready for review
November 20, 2026 11:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces #182.
/integrations/integration-toolkit/large-data-loads): how migrations, full resyncs and large file imports are recognized and processed, what an integrator notices, and what a client has to handle:503withRetry-After: 60andBULK_CAPACITY_EXHAUSTEDwhen the capacity for large loads is in use. Nothing of the request is queued; in a mixed request the interactive events are accepted and come backskippedon the retry.503withRetry-After: 2andINGEST_THROTTLEDwhen the allowance cannot be checked or the receiving queue throttles. Nothing was accepted, the same request is sent again, no duplicates.422withEVENT_TOO_WIDEabove 5,000 updates per event andREQUEST_TOO_WIDEabove 25,000 updates per request. The page says plainly that staging and production only observe these limits today.progress.waiting_for_capacity)./integrations/integration-toolkit/sandbox-organizations), rewritten from docs(integration-toolkit): sandbox organizations and the SANDBOX_LANE monitoring code #182 for the new behaviour: everyday traffic is handled as in production, large loads in a sandbox share a small capacity and run at a low rate. The Integration Hub banner and theSANDBOX_LANEnotice link to this address.inbound/getting-started.md: a "When to Retry" table for429and the two503responses, and a pointer to the size limits.interactive-inbound.md:503added where retries are described; interactive events do not count towards the allowance and are never refused for lack of capacity.monitoring/investigating.md: a replay can be answered with503, or with200andRetry-Afterwhen it was throttled part of the way.overview.md,configuration.md: cross-links.monitoring/codes.mdis generated and is not changed here. The new monitoring codeEVENT_TOO_WIDEarrives there with the nextnpm run update-monitoring-codes.Status of the feature. It is implemented and merged in the Integration Toolkit, tested on dev, and switched off on every stage. It is not in production. Both pages carry a "Not active yet" note and describe what happens once it is on, without a date.
Do not merge before the feature is live, or decide first that the "Not active yet" notes are enough to publish ahead of it. The notes have to be removed when it is switched on, and the note on the size limits when they are enforced in production.
Checked:
npm run buildpasses, and the anchors the pages link to exist in the built page.Update 2026-11-20: bulk import windows, and large loads are live
GET/PUT/DELETE /v1/bulk-imports/window). Planning a Migration now recommends booking one.Merge only after the API (integration-toolkit-api !333) and the Hub card (epilot360-integration-hub !91) are in production.