feat(fetch): retry uncharged busy answers so a brief capacity wait does not fail the command

[gonzalo-zenrows]

Why

zenrows fetch and zenrows extract sent each request once. When the gateway was briefly busy (429, 502, 503, 504, none of which is charged) or the network dropped, the command failed. A 503 RESP009 ("at capacity, retry shortly") fell into the generic branch that told agents to rerun with --manual --js-render --premium-proxy, the most expensive configuration, which does not help.

This applies the retry policy the Zenrows Fetch/Extract clients share.

What changes

• Retries. 429, 502, 503, 504 and network errors before any response are retried up to --retries <n> times (new flag on fetch and extract, default 2, 0 sends once).
• Delay. Retry-After in whole seconds, capped at 30 s. Otherwise full jitter: uniform(0.25 s, min(30 s, 1 s × 2^attempt)), with attempt the 0-based retry index (first retry ≤ 1 s, second ≤ 2 s).
• Budget. No retry starts when the time already spent plus the next delay would pass 90 s.
• Never retried: the client timeout (--timeout; the API keeps working and may still charge a 200), any other 4xx, and a 429 whose body says the account is out of credits (it answers the same until the account changes). The EXTRACT_DOMAIN_NOT_ENABLED fallback to --autoparse is unchanged; a test pins that it spends no retries.
• SERVICE_BUSY. A new error for 503 RESP009, with server_code, retry_after, a likely_cause that says the request was not charged (for Extract: the extraction service is still preparing this layout), and a next_action to retry the same command shortly without escalating.
• Gateway budget. SERVER_BUDGET_MS is now 180 s (the whole request: auth, fetch, parse), not 90 s. The REQUEST_TIMEOUT error now says the API may still finish and charge the request, and suggests a --timeout above 180 s.
• Shared helper. Retry-After parsing and the delay schedules move to src/core/backoff.ts. Crawl uses it with its own schedule unchanged (250 ms × 2^attempt, capped at 10 s, ±20%).
• Docs. --retries in the fetch/extract help, the README Fetch section, and the protected-fetch and trace-debug skills.

Decision for the reviewer

The default client timeout stays at 120 s, below the gateway's 180 s budget. The old test pinned "client timeout above server budget" so the API could always answer with its own error. That no longer holds, and I replaced the test rather than raise the default, because changing the default timeout was out of scope. Raising it above 180 s (for example to 200 s) would restore that property. It is a one-line change if you want it here.

Testing

npm run typecheck && npm test: 402 tests, 0 failures (Node 26 locally; CI runs Node 20 and 22). New tests/fetch-retry.test.ts uses a stubbed fetch and an injected sleep and clock. It covers: jitter bounds, including on 503; Retry-After honoured and capped; the 90 s budget; no retry on 400/401/404/422 or on a timeout abort; the default of 2; --retries 0; flag validation; the unchanged autoparse fallback; and the SERVICE_BUSY shape for Fetch and Extract.

Not verified against the live API.

[gonzalo-zenrows]

Same retry policy across all clients (429/502/503/504 and pre-response connection errors; 2 retries by default; full jitter; Retry-After capped at 30 s; 90 s total budget; timeouts never retried):

• Python SDK: feat(client): retry only failures another try can fix, with jittered backoff zenrows-python-sdk#62
• Go SDK: feat(api): retry transient failures by default with full-jitter backoff zenrows-go-sdk#43
• Node SDK: feat(client): retry uncharged failures by default, with full-jitter backoff and a time budget zenrows-node-sdk#58
• CLI: feat(fetch): retry uncharged busy answers so a brief capacity wait does not fail the command #64
• MCP: feat(fetch): retry not-served scrape and extract calls so a busy API does not fail the tool call zenrows-mcp#80