Skip to content

experiment: GitHub digest on PyGithub, Crowdin scripts on crowdin-api-client - #63

Closed
Bilb wants to merge 1 commit into
feat/contributor-pr-digestfrom
experiment/service-sdks
Closed

Bilb wants to merge 1 commit into
feat/contributor-pr-digestfrom
experiment/service-sdks

Conversation

@Bilb

@Bilb Bilb commented Sep 25, 2026

Copy link
Copy Markdown
Collaborator

Experiment, stacked on #59: replace the hand-rolled HTTP for GitHub and Crowdin with their SDKs while keeping this repo's retry, error handling, permissions and test fakes. SOGS, Discord and the Claude CLI are deliberately untouched.

What the SDKs do and do not give us

crowdin-api-client 1.29.0 never retries a 429 (its should_retry is false for 300–499) and retries 5xx with a fixed 100 ms sleep. crowdin/crowdin_sdk.py swaps the requester's session for one routed through shared.retry and turns the SDK's own loop off, so the 5/10/6-attempt budgets, Retry-After handling and FakeSession tests are unchanged. Per-thread sessions moved into that shim, so the download and approve scripts get them as well as the report. with_fetch_all() replaces paged().

PyGithub 2.10.0 — GithubRetry is configured to the same shape as shared.retry (6 attempts, backoff to 30 s, Retry-After capped at 60 s, 429 + 5xx) and adds the 403 secondary rate limit. Two quirks found while porting, both handled:

  • PaginatedList.totalCount on a search is read off the last page link, which GitHub caps at 1000 (it reported 1000 for a query GitHub counts at 48,951), so truncation now comes from the item count.
  • Search results lazily fetch any attribute missing from the raw JSON, one GET per PR. Production payloads carry everything the digest reads; the test fixtures now must too.

Also: advanced_search=true is no longer required — GitHub answers 200 either way — so that comment was stale.

Verification

  • shared / github_prs / crowdin suites: 63 / 41 / 24, pyflakes clean.
  • digest.py --dry-run --window-hours 720 against the live org: payload byte-identical to the requests version (6 new PRs in both), 4 requests instead of 3 (the extra is GET /orgs/{org}), no per-PR fetches.
  • Crowdin has no live run (no token on this machine); covered by tests only. approve_strings.py has no tests and is compile-checked only.

Cost

+269/−229. New dependencies: crowdin-api-client (+Deprecated, wrapt), PyGithub (+pyjwt, pynacl, cryptography, typing-extensions). The GitHub tests need a ~25-line fake of PyGithub's connection interface instead of FakeSession, and PyGithub's retry is not unit-exercisable through it.

My read: Crowdin is a clear net gain; GitHub is roughly a wash.

…-client

Both keep this repo's retry, error handling and test fakes; the SDKs supply the
endpoints, parameter names and pagination.

crowdin-api-client never retries a 429 and retries 5xx with a fixed 100 ms
sleep, so crowdin_sdk.py swaps its requester's session for one routed through
shared.retry and turns the SDK's own loop off. Sessions are per thread there,
which the download and approve scripts now get as well as the report.

PyGithub's totalCount on a search is read off the last-page link, which GitHub
caps at 1000, so truncation comes from the item count. Search results lazily
fetch any attribute missing from the raw JSON, one GET per PR, so the test
fixtures carry every field the digest reads. The dry-run payload against the
live org is byte-identical to the requests version, at one extra request
(GET /orgs/{org}).
@Bilb Bilb closed this Sep 25, 2026
@Bilb
Bilb deleted the experiment/service-sdks branch September 25, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant