Conversation
One message each weekday morning listing the open PRs across session-foundation's own repositories whose author is not a maintainer and which have moved in the last three days. A single search fetches every open PR in the org and the window is applied to the result, so the header can carry the total contributor backlog for the cost of one query. Forks and archived repos are excluded by checking results against the org's repository list rather than by name, and bot accounts on GitHub's own account type. Weekdays means Monday has to cover the weekend, so the window is 72h and overlaps itself by two days on every run. A state file absorbs the overlap: 🟢 is a PR never reported, ✏️ one that has moved since it was, and a PR that has not moved is left out. Only what Discord accepted is recorded, so a run that fails partway re-reports that message's PRs rather than losing them, and every way of failing to read the file treats the window as new — noisy once, never wrong. Dedup is keyed on updated_at, which moves on any change at all, so an edit touching several PRs at once resurfaces all of them. The accurate alternative needs a request per PR that moved; this is the cheaper half of that trade. Maintainers are a hand-written list. Org membership covers six accounts, two of which are not in the review loop, and push access is held by a dozen more as outside collaborators — several of them contractors whose PRs are the point of the digest. Both signals would get it wrong in both directions. Runs on the box that already hosts the Zendesk digest, under its own user, env file and venv — the venv because the two jobs pin requests differently.
A block over the text budget was posted whole and rejected, and since nothing in a rejected message is recorded, the same block was rebuilt on every run until the PRs aged out of the window. Blocks are now split under a repeated heading, each sized to fit beside the header.
…s in the docs Exclusion is by repository property, so nothing private is named anywhere in the repo. The token guidance drops the option of a `repo` scope with it: the digest posts to Discord, and nothing about a private repository belongs there.
…eads Forks, archived and private repositories are always left out; there was no run that wanted them back in.
The envelope's `result` carries the reason whichever way the CLI exits, and the exit-1 path already reads it. The exit-0 path reported only the subtype and the status code, which does not say what went wrong.
…ers into shared/ The pull request digest was about to carry its own copy of each; crowdin's report already does. One copy, imported by every script out of the same clone. triage.py keeps get_env, request_with_retry, clip and post_to_discord in its namespace: note_reply, resolve_reviews and the alert reach them there, and the tests patch them there.
The digest carried its own copy of the retry loop, the dedup state file, the Components V2 constants, the chunker and the webhook posting. It now imports them from shared/, and keeps only what is its own: the record it stores per PR, its per-message block cap, and the rendering. The tests that covered the copies move with the code; what stays here covers the wrappers and the rendering.
…imeout RFC 9110 allows Retry-After in either form. Crowdin's report already parsed the date form, so the shared loop has to before it can replace that copy; a date already gone reads as no header, since time.sleep() rejects a negative. The timeout was fixed at 30s; Crowdin's scan runs on 60.
…k code The third copy of the retry loop and the Discord posting. What stays is Crowdin's own contract: ten attempts at a 60s timeout, a 4xx that raises so callers can read the body unchecked, and the plain-text warning posted before the run exits on a rejected embed.
…ponses Each golden replays a recording of API responses through the unchanged script and compares its output byte for byte, so the packaging and transport changes that follow cannot alter a payload unnoticed. - digest: a live --dry-run over a 720 h window (2026-09-25), trimmed to the fields the digest reads; its replay is byte-identical to the live output. A second case adds a state file, covering the new/changed/unchanged split. - report: --locales de over a synthetic project with duplicates in plain and plural slots, both the Discord payload and the --json findings. - download: the export payloads and the files written, invoked as the sync workflow invokes it. RecordedSession matches requests by method, URL, query and body rather than order, because both Crowdin scripts fan requests out across threads.
One job per suite, since two requirements files pin requests differently. sogs_moderation runs on the system interpreter with a venv that can see python3-session-util, which ships as a deb rather than a wheel. ruff is limited to pyflakes and syntax errors, the class of mistake a large refactor introduces. The one finding, an unused import in relay.py, is removed.
…ails OnFailure= reports a run that failed and nothing about one that never happened: a timer left disabled, a unit renamed, a host down through a whole schedule. Each digest unit now touches /var/lib/session-ops/stamps/<name> from a `+` ExecStartPost=, which a oneshot runs only after every ExecStart= succeeded and which runs outside the sandbox, so neither job gains a write path. silence.py, on an hourly timer under an account of its own, compares each stamp's age with deploy/jobs.toml and posts one message for every job past its max_age_hours, repeated daily while it stays quiet. A missing stamp is timed from the first check that found it missing, so installing the checker alerts on nothing. The stamps are root's and world-readable, so the checker needs no access to the Zendesk state directory and the ticket data in it. test_silence.py fails when a shipped timer has no registry entry or no stamp line.
pyproject.toml and uv.lock replace the five requirements files, which pinned
requests two different ways and so needed a venv each. The code moves to
src/session_ops/{shared,github_prs,zendesk,crowdin,sogs,monitor} with absolute
imports, so no module inserts anything on sys.path; tests move to tests/ in the
same shape and run as one discover.
Every script is a console entry point, and the units call those out of one venv
at /opt/zendesk/.venv, built with the system Python: a uv-managed interpreter
would sit under /home/zendesk, which ProtectHome=yes hides from the units that do
not run as zendesk. test_units.py fails when a unit names an entry point the
package does not declare. The relay runs note_reply with -m rather than by path.
download_translations_from_crowdin.py parsed its arguments at import; that moves
into main(argv) so it can be an entry point. babel stays pinned exactly, since its
CLDR data is what the generated language names come from.
CI runs one locked environment instead of a matrix. tests/sogs skips itself
without session_util; its own job checks the import first, so the skip cannot hide
a broken suite. The Crowdin workflows install the package and run modules until
they move to the host.
All 586 tests and the four goldens pass unchanged, and a live digest dry run
through the new entry point matches its golden apart from PR ages.
shared/http.py replaces shared/retry.py. Session is a requests.Session with a urllib3 Retry mounted, so retries happen below the caller, which sees only the final answer: 429 and every 5xx retried for all methods, 1 s doubling to 30 s, Retry-After in either form capped at a minute, GitHub's x-ratelimit-reset when Retry-After is absent, and an unusable header falling back to backoff. It adds a default timeout, an optional token bucket, and a per-call attempts= for optional data, carried to the adapter in a ContextVar because requests has no way to pass one down. Its tests run against a local server, so the retries tested are urllib3's own. The Crowdin scripts move to crowdin-api-client through crowdin/sdk.py. The SDK never retries a 429 and retries 5xx on a fixed 100 ms, so its session is swapped for per-thread http.Sessions drawing on one 30/s bucket, and its own loop is off. Its decoder also turns every ISO timestamp into a datetime, which json.dump cannot write: the download's project and glossary files would have failed on the first real payload. Responses stay plain JSON. The recordings now carry timestamps as Crowdin sends them, and approve_strings.py gains a golden. Every golden was regenerated by the previous code from the same recordings, and the new code matches it byte for byte; only request keys the SDK spells differently (offset=0, a query moved into params) were edited. A live digest dry run matches its golden apart from PR ages. Caller tests that queued a retry's worth of failures now queue one, and assert the budget they ask for instead.
… daily The sharded daily scan saw each locale once every eight days and could say nothing about slots that had been resolved. This keeps the open slots, keyed (string, locale, plural category), and posts only what changed. - crowdin-relay.service takes Crowdin's suggestion events, acknowledges at once and re-checks the one (string, locale) named. Crowdin signs nothing, so the secret is the last path segment, and a wrong or unset one is a 404. It is a process and account of its own rather than a route on the Zendesk relay, which can write public comments and has no business holding the Crowdin token. - crowdin-reconcile-duplicates, daily, judges every string of every locale and posts new and resolved slots, or nothing. Crowdin never retries a webhook, so this is what keeps the state correct. --croql narrows each locale to the strings CroQL counts two translations for; it stays off until that query is checked live. - Both write the state under a file lock. The relay records when it checked each (string, locale), and a scan's older view of that scope is ignored, so a scan that began before a suggestion landed cannot resolve what the event opened. - The state is written only once every message landed; a run that fails to post repeats its whole diff next time. --seed records the current slots without posting them. report_multiple_translations.py and its workflow stay until a reconciliation cycle has run clean on the host; the two now share the slot logic, and its golden is unchanged.
Each job's section of the README moves to docs/jobs/<unit name>.md, opening with where it runs, which secrets it reads, how to dry-run it, how to re-run it and where it logs. The text moves as it was; only headings and relative links change. The README becomes the index, and tests/test_docs.py fails when a job in the registry has no page. crowdin-duplicates and session-ops-silence are new pages; the SOGS tools, which nothing schedules, move to docs/tools/. crowdin-approve-strings is the only script that writes to Crowdin, and it read the same token as every read-only job. It now reads CROWDIN_PROOFREADER_TOKEN, or the keyring's proofreader-api-token, and never falls back to CROWDIN_API_TOKEN, which can therefore be read-only everywhere else.
…iled src/session_ops/jobs.toml is the registry: each job's entry point, arguments, account, env files, required variables, schedule and allowed silence. From it, `session-ops units` writes each job's drop-ins over deploy/session-ops@.service and .timer, which hold the hardening once; `session-ops run <job>` runs a job the way its timer does; and the silence checker watches every scheduled one. A run checks the variables its job needs, gives it a scratch directory in the unit's private /tmp, and reports its own failure: the job, the host, the step it was on, one sentence of error with secrets and webhook URLs scrubbed, each target's result for a job with several, and the commands to read the journal and re-run it. The traceback stays in the journal. Having alerted, it records its invocation id, and the OnFailure= backstop stays quiet for that invocation, speaking only for what a run cannot report: killed, timed out, Discord unreachable, and the relays. ALERT_DISCORD_ROLE_ID, which the GitHub notifier mentioned, is mentioned by both. The Zendesk resolver and digest become one job with two targets, the resolver optional: its failure is reported and the digest still runs and succeeds, which is what the `||` in the old unit did. The tests that read that unit now assert the same order and failure handling on the job itself. deploy/install.sh replaces the numbered install steps and is idempotent: accounts, venv, env files created empty at 0600 (systemd reads them as root, so no account can read another's), tmpfiles, units and drop-ins, then it enables each job whose env file has content. It moves the previous layout's env and state files across where the new ones do not exist yet, and retires the per-job units. Code now lives in /opt/session-ops, owned by root; kept run copies under /var/lib/session-ops/<job>/runs are pruned after 14 days by tmpfiles.
crowdin-sync is one process where the workflow was eight jobs passing artefacts: download, parse and validate, then Android, iOS and the localization module as independent targets, so one failing to publish does not stop the others and the alert says which. Each platform is a shallow, sparse checkout of only the paths its generator writes. It keeps today's publishing: a pull request from feature/update-crowdin-translations rebuilt from dev and force-pushed, and a commit straight onto session-localization's main that overwrites the generated files without removing others, which is what the workflow's misdirected `rm` amounted to. The Gradle validation step is gone; Android's CI checks the pull request. The run's downloads, parsed JSON and validation report are kept under runs/ for 14 days. snode-list copies the dynamic-assets list into session-ios, byte for byte and only once it parses as JSON. release-stats is the TypeScript script in Python, on demand; fed the same live releases, both wrote byte-identical CSVs, which are now its golden. Node leaves the repo with it. Publishing replaces peter-evans/create-pull-request and git-auto-commit-action with git and the REST API: an unchanged tree is not pushed again, and nothing left to merge closes the pull request and deletes the branch. It authenticates as a GitHub App when one is set up, minting a one-hour token scoped to the repositories the run pushes to, and otherwise with GITHUB_PUBLISH_TOKEN, the PAT the workflows used. The App's key reaches only the unit, through LoadCredential=. The token reaches git in GIT_CONFIG_* variables, never on a command line. The workflows these replace keep their schedules until the host has run each twice.
A dry run printed an empty stat: it diffed the index against HEAD after committing. It now shows the commit. A failed git command quoted the first line of stderr, which is whatever a wrapper or credential helper printed first. It now quotes git's fatal: or error: line.
Checked live on 2026-09-25. Across all 80 locales CroQL returned only plural strings, which it cannot tell from duplicates, until a second suggestion was planted on a singular string in fr: that string was then the one singular candidate, and the narrowed check and a full scan of fr found the same slot. A locale now costs about 50 requests instead of 1,371.
Found by installing and running every unit in a systemd container.
- The OnFailure= backstop quoted the unit's last journal line whatever the run: a
run killed by its timeout before logging anything was reported with the previous,
successful run's "Written to ...". It now reads only the failed invocation, and says
how the unit failed (timed out, killed, out of memory).
- An error was cut at its first newline, so GitHub's pretty-printed 401 body read
as `{`. Errors are now whitespace-collapsed and clipped.
- A Crowdin error read as the SDK exception's repr; it is now "Crowdin 401:
Unauthorized".
- A job that names no step no longer fails "during starting".
…y the digest note_reply reaches it too, and has no --batch-size to lower.
…t of triage.py
note_reply, resolve_reviews and the failure alert imported the 1,900-line
classifier to reach a ticket fetch, a marker or the CLI's name. Those now live in
modules of their own:
zendesk/api.py the session, search and its 1000-result ceiling, a ticket,
its comments and users, requester activity, the markers,
the customer's side of a conversation, store reviews
zendesk/claude_cli.py the CLI call, the model aliases, a failure worth quoting
zendesk/transcript.py the English transcript: detect, render, write, attach
shared/text.py undash_english and squash, which name no service
triage.py keeps the queries, the taxonomy, the dedup, the review filter, the
analysis and the rendering: 1,160 lines. resolve_reviews and the alert no longer
import it at all.
Nothing is renamed or rewritten. Each of the 112 top-level definitions in the old
triage.py is in the new tree with its source text unchanged, checked by comparing
every definition's source segment before and after. Callers and tests change only
which module they name, and a patch now lands on the module where the name is
looked up. Against the live Zendesk account, the previous commit and this one
produce a byte-identical --dump-batch and the same resolver dry run.
…ir own A command's read, the transcript's and the description hydration's differed only in page size, order, retry budget and what a failure does. Those are now fetch_comments' parameters; optional= is the enrichment's budget and failure mode, a note and None rather than an exit. Each caller sends the request it sent before: test_api.py pins all three, and passes against the previous commit too. The one visible change is that hydration now prints a note when Zendesk answers it with an error, where it skipped the ticket silently.
note_reply passed --model to the CLI as given, so `opus` meant whatever the CLI calls opus that week, while the digest maps the same word to a pinned id. Both go through resolve_api_model now; a full id still passes through.
…n the quota is spent GitHub answers a rate limit with a 403 as often as a 429, so the transport never retried it. It also sends x-ratelimit-reset on every response, so a 5xx without Retry-After waited for the rate-limit window, capped at a minute per attempt, instead of backing off. The reset is read now only when x-ratelimit-remaining is 0, and a 403 is retried only when it carries Retry-After or an exhausted quota.
…mplete search as a floor The search is sorted by update time, so a PR updated between the two page fetches moves onto page one after it was read and pushes that page's last item onto page two: that PR was listed twice and the updated one missed. Results are now deduped by id, and a search GitHub flags incomplete_results reads as a floor like the 1000-result ceiling does. The page loop is bounded by the ceiling rather than by how many items have arrived, which a repeating page would never grow.
…eads as updated State entries were pruned 30 days after a PR was last reported, so a contributor PR that went quiet for a month and then got a push came back as 🟢, which the docs define as never reported. The default retention is now 365 days, a few KB of state, and the docs say what 🟢 means under it.
The property the state file exists for lived only in main(), which no test ran past a dry run. These run main() from the fetched PRs to the state file with Discord faked to accept all, some or none of the messages.
When Discord was unreachable, post_to_discord printed the exception verbatim, and urllib3's text quotes the request path, /api/webhooks/<id>/<token>. That line became the journal line the OnFailure backstop quotes, and the scrub missed it because it only matched a full https://discord.com URL. post_to_discord now prints only the exception's type, the scrub also matches a bare webhook path (with or without an API version), and the backstop passes the journal excerpt through the same scrub before quoting it.
Every unit runs /opt/session-ops/.venv, but install.sh took its root from wherever it was run. Run from an old /opt/zendesk clone it removed the old units and installed new ones that all fail with 203/EXEC, and said nothing. It now stops unless it sits in /opt/session-ops, and unless that clone is owned by root, since the units run its code.
…ale jobs install.sh copied /var/lib/zendesk/seen.json and the other old state before it disabled the units that write it, so a digest running at that moment wrote state the new layout never saw. The old timers are now disabled and their services stopped before anything is copied. A timer also stayed enabled after its job left jobs.toml or its env file was emptied. Every enabled session-ops@ timer that `session-ops list --ready` no longer names is now disabled, which keeps a rerun idempotent.
On a migrated host alerts.env is created empty, and the OnFailure backstop, which loads only that file, then exits without reporting anything, while the silence checker is never enabled. install.sh now ends with a banner saying so until ALERT_DISCORD_WEBHOOK_URL is set.
The deployment guide had grown to repeat rationale the code, the units and docs/jobs already carry. It now keeps the instructions: requirements, install, migration, each env file, the relays and nginx route, verification, updating and recovery. It also corrects what it said. The dry run of the pull request digest does not run under the unit's confinement, and a second run logs "0 new, 0 changed since last reported, N unchanged". Migration now tells the operator to fill alerts.env, without which the backstop and silence checker are off. Publishing needs the GitHub App (GITHUB_APP_ID and github-app.pem); a token is no longer documented. Crowdin reconciliation, like the relay, refuses to run until the slots are seeded. The pull request digest catches up after an outage; the Zendesk digest's 72-hour window does not, so the guide says what an outage longer than that loses and how to re-run with a wider window.
…record that is not an object load_state promises that every read failure is a cache miss, but invalid UTF-8 raised UnicodeDecodeError past its handler and failed every run until someone deleted the file. Catching ValueError covers it along with bad JSON. save_state runs after the digest has posted, so a stored record that is not an object (hand-edited or from an older writer) raised AttributeError at the worst moment: the post had landed but the state was not written, and the next run reposted. Such a record is now dropped like any other malformed entry.
PUNCTUATING_DASH matched \s, which includes newlines, so undash_english folded a dashed list or a dash before a sign-off into the line above it: 'Try these:\n– restart the app' reached the customer as 'Try these:, restart the app'. Only spaces and tabs now count as the dash's surroundings. That alone would turn each list bullet into a leading comma, so a dash that opens a line becomes the hyphen bullet a person would type, and a dash that ends a line leaves a comma with no trailing space.
…assification does The digest classified with resolve_api_model(model) but handed attach_english the model as given, so an alias such as opus meant a pinned id for the classification and whatever the CLI currently calls opus for the transcript. Both now use the resolved id.
… author
fetch_user already turned an HTTP error into {}, but a connection error or an unreadable body raised straight out of it. Through customer_authors that escaped attach_english, whose contract is never to raise, and failed the whole digest over one translation. An unresolved author already counts as a customer there, and note_reply's command check refuses an unknown user, so {} is safe for every caller.
…t path Every ticket the resolver solves gets a private note naming the script that solved it, so an agent can find out why. It still said zendesk_triage/resolve_reviews.py, which no longer exists. The note is now a module constant, and a test checks that the path it names is this module.
…t modules The relay docstring's usage line ran uvicorn relay:app, which only worked from the old flat layout; the unit runs session_ops.zendesk.relay:app. The resolver doc named triage.is_store_review, which lives in api now and is shared by the triage and the resolver.
sogs-ban's docs and its missing-module message sent people to deb.oxen.io, while CI installs the package from deb.session.foundation. One repository is named everywhere now, the one CI proves works.
… git command that failed A push GitHub refused alerted with git's closing 'error: failed to push some refs to <url>', which says nothing about why. The reason is in the 'remote: error:' line (GH006 on a protected branch) or, failing that, the ' ! [remote rejected]' line, so reason() now prefers those over fatal:/error:. A failed commit was labelled 'git -c failed', because the commit passes its identity as -c pairs ahead of the subcommand. The label now skips those pairs.
… out Each retry's sleep is capped at a minute, so a Retry-After or an exhausted quota's reset further off than that was retried anyway: every attempt slept 60 s and hit the same spent quota. With the default six attempts that is 300 s, release-stats' whole unit timeout, so systemd killed the job before it could report the rate limit. When the server's wait exceeds a minute for each retry left, the response now comes back immediately, the way a 403 without rate-limit headers already does. A wait the budget can cover is still slept, a minute at a time.
… when it cannot Publishing fell back to a personal token, GITHUB_PUBLISH_TOKEN, whenever the App id or its key was absent, so a half-configured App silently published as a person with a long-lived token. The App is now required: without GITHUB_APP_ID, or with the github-app.pem credential empty or absent, a run exits naming which of the two it lacks and where the key is read from. install.sh still creates the key file empty so the units can start, and that message is what then says why a run failed. The App branch had no test of its own; one now mints through installation_token with a real signed JWT and checks the owner and repositories it asks for.
…p longer than its window loses nothing The contributor PR digest applied a fixed 72-hour cutoff from the time it ran, whatever happened before. Anything that moved in a longer gap was dropped without a trace: April's DST weekend is 73 hours between Friday and Monday 09:30 in Melbourne, the timer's two-minute random delay stretches most Mondays past 72 hours (and the cutoff was taken after the search), a host down across a run catches up with the same cutoff, and a Friday post that fails partway leaves PRs Monday's window no longer reaches. The state file now records covered_until: when the last run whose every message Discord accepted started its search, taken before the search so a PR that moves during it falls in the next window. Each run reaches back to the earlier of that and the configured window, and no further than the state retention, past which the dedup has forgotten what it reported. A partial post stores its own cutoff instead of advancing, a dry run writes nothing, and a state without the field behaves as before. The header names the span actually covered. shared.state.save_state takes the field as an optional extra; without one the file is exactly what it was, so the Zendesk digest's state and window are unchanged.
…uired release-stats writes its CSVs to its own state directory and never publishes, and the key comment in jobs.toml still described the token fallback that is gone.
Since fetch_user returns {} for a lookup that fails in transport, latest_command read an unreachable /users endpoint as an unauthorised author: it printed "Ignoring a command from an unknown user", and main then cleared claude-queued, so the agent's command was dropped with nothing on the ticket to say so.
An author that cannot be looked up now fails the run instead. The tag stays, the relay logs the failure, and the ticket remains in the queue view. A lookup that answers with a role the command check refuses still stops the search as before.
Both relays restart two seconds after dying, and systemd's default limit is five starts in ten seconds. A relay that takes half a second or more to die, such as uvicorn failing at import, never packs five starts into ten seconds, so it restarted forever without reaching the failed state OnFailure alerts on. Five starts in five minutes trips on any such loop within about a minute of it starting.
install.sh checked only that /opt/session-ops itself was owned by root. A clone made as a normal user and then given a non-recursive chown root, which is what the old message invited, passed while every file stayed writable by that user, and root then ran uv sync and session-ops from it, as does every job. It now refuses any path under the tree that is not owned by root or is group- or world-writable, ignoring symlinks, and says how to fix it.
The secret is the last segment of the Crowdin route's path. access_log off kept it out of the access log, but an upstream error (the relay down or restarting) logs the full request line to /var/log/nginx/error.log, which the adm group can read. That route's errors now go to /var/log/session-ops/crowdin-relay.error.log, in a directory tmpfiles creates as root-only, keeping the error-level detail for debugging.
…re a scan The relay checked that the state existed before its Crowdin requests, outside the lock, and load() then returned an empty state for a missing file. A state moved aside in between (what a version mismatch tells the operator to do) was recreated nearly empty, and the next reconciliation, whose only guard was that the file exists, posted every open slot as new. load() now refuses a missing file unless the caller passes missing_ok, which only --seed and --dry-run do. Reconciliation still refuses before scanning rather than after.
StateDirectory= defaults to 0755 and jobs wrote their files under umask 022, so every account on the box, the internet-facing relays included, could read every job's state; save_state's rewrites also undid install.sh's 0640 on migrated files. State directories are now 0711 and jobs run with UMask=0077. Not 0700: the OnFailure backstop runs as sessionops and opens each job's alerted marker by name, so the directory stays traversable and the runner makes that one file readable. crowdin-relay shares crowdin-duplicates' directory and sets the same mode, since systemd reapplies it at every start.
The fetch only checked that the source parsed as JSON, so an upstream [] or {}, or a list regenerated wrong, would open a session-ios pull request replacing the fallback list with nothing, leaving PR review as the only gate on the file that rescues clients who cannot reach the seed nodes.
It now requires a non-empty service_node_states whose every entry has a public_ip and a pubkey_ed25519.
sparse_clone runs git clone from the parent directory, which resolved a relative destination against that parent again: work/session-ios landed in work/work/session-ios and the sparse checkout then failed. The runner always passes an absolute directory; a hand run with a relative SESSION_OPS_WORK_DIR did not.
The usage text called the bare invocation what the timer runs, contradicting the next example; the timer passes --window-hours 72 and --state.
A full --croql run against the live project took 509 s across 80 locales, not the 20 minutes stated, and a locale scanned without CroQL took 62 s, about 85 minutes for all of them rather than an hour.
container_message sent no allowed_mentions, so Discord parsed every mention in a Components V2 message's text. The PR digest quotes titles and logins written by outside contributors, so a PR titled "@everyone please review", or opened by an account named everyone or here, pinged the whole channel every weekday it moved. The Zendesk digest had the same gap with ticket text. The runner and backstop alerts already sent {"parse": []}. Every container message now sends it too. Nobody needs pinging from a digest; the channel is read when people can.
A contributor could title a PR "[Download the build](https://example.com)" and the digest rendered it as a masked link carrying the digest's authority. Square brackets in titles are now escaped, so a masked link shows literally, and every URL is wrapped in <> so Discord shows it plainly without a preview.
The first real runs of crowdin-sync and snode-list would otherwise push to the bot branches the GitHub workflows still own and to session-localization's main, and zendesk-digest would solve live tickets, so the only way to see what the host does was to let it do it. While /etc/session-ops/rehearsal exists, the runner marks every job as a rehearsal. Pull requests then come from rehearsal/ branches with a [Rehearsal] title and a do-not-merge note, a direct push opens a pull request against its branch instead, and the Zendesk resolver only reports. deploy/README.md covers the rest of a rehearsal host's setup and the clean-up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two changes in one PR, because the second replaces most of the first's scaffolding before either has reached the host. They were #59 and #65; #59 is closed into this one.
tests.ymlonce the host has taken over.It reviews in commit order, one phase at a time.
The contributor PR digest
Each weekday morning, one message lists the open PRs across session-foundation's own repositories whose author is not a maintainer and which have moved in the last three days.
🟢 is a PR the digest has not reported in the past year. ✏️ is one it has, which has moved since. A PR that hasn't moved is left out of the message but still counts in the backlog line.
Mon..Fri 09:30 Australia/Melbourne, so Monday's run has to cover the weekend: hence a 72h window, which overlaps itself by two days.--stateabsorbs the overlap. It records which PRs reached Discord and each one'supdated_at. Only what Discord accepted is recorded, so a run failing on its second message re-reports that message's PRs. Any failure to read the state file treats the window as new.updated_atalone. That is a deliberate trade: a label sweep resurfaces every PR it touched. The accurate alternative, head SHA plus comment counts, costs a request per PR that moved.incomplete_results, or when a PR updated between page fetches shifts the pages.Operations and flags are in
docs/jobs/github-prs-digest.md.Phases
ae60e0d…36702d3edbe5a6,d2510e569433f7session_opspackage:pyproject.toml+uv.lock,src/layout, entry points, one venv7633a36shared/http.py: urllib3Retry+ token bucket); Crowdin oncrowdin-api-cliente2fa9ce226047adocs/jobs/(its proofreader token went withcrowdin-approve-stringsin14cccbe)209bc03jobs.tomlregistry,session-ops run, one templated unit, three-layer alerts,install.shf1cdc0ed93a4194b61cf2,807b0a0,3a9ba47triage.pysplit intozendesk/api.py,claude_cli.py,transcript.py: a verbatim move, then the comment-fetcher merge and the--modelalias fix on their own1f1d4af,20f0e98,92de8b4,29af04f5a54b28…f13a601b5c9128…048a2abda72a72…2dfba56How it was checked
Goldens. Every golden replays recorded API responses and compares output byte for byte. Each Crowdin golden was regenerated by the previous code from the same recording before the new code was held to it.
The digest against the live org and a test webhook:
A busy-day render split into two messages, 3830 and 743 characters, on a repository boundary. The digest's replay is byte-identical to a live
--dry-run, before and after the packaging and transport changes.Release stats. In Python, fed the same live releases as the TypeScript it replaces,
release-statswrites byte-identical CSVs. Those CSVs are its golden.Snode list. A live
session-ops run snode-list --dry-runfetched the list, sparse-cloned session-ios and found the change to publish.systemd. The unit template and every generated drop-in pass
systemd-analyze verify, and each schedule passessystemd-analyze calendar.Publishing. It is tested with real git against bare repositories on disk.
Crowdin, live and read-only. The CroQL narrowing caught a planted duplicate in
frthat a full scan also found, and found nothing anywhere else. That matches the old report's last full cycle.crowdin-sync --dry-rundownloaded and generated all three platforms. The localization module has no changes againstmain. Android and iOS differ from the workflow's own bot branches only by that planted translation.Install.
deploy/install.shran in a systemd container on Ubuntu 24.04: a fresh install, a second run once the env files were filled, then an idempotent third. Every job started under its own account. Failures alerted once from the run while the backstop stayed quiet, and the backstop reported a forced timeout. Both relays answered correctly under their sandboxes.Against main
Every job that moved was run on both sides against the same live data, with nothing pushed or posted.
main's side replays each workflow step by step with its own scripts, including the artefact hand-offs between jobs; this branch's side runs each job with--dry-run. Both started from the same commit of each target repository.frduplicate and 3 inkmr--dump-batchWhat the patches do not show:
github-actions[bot]; the jobs commit asPUBLISH_GIT_AUTHOR.strings.xmlstays;crowdin-syncdeletes it.generatePlayDebugResourcesbefore opening the PR;crowdin-syncleaves validation to session-android's own CI.Second review
5280335); an optional target's failure whose alert can't post fails the run (2329cde); webhook tokens stay out of the journal (69707f7); no role mention anywhere (c531da0)./opt/session-ops(7cdd6ff), stops old units before copying their state and unschedules stale jobs (04f9c31), and warns whilealerts.envis empty (f2b21e7).b5c9128); reconciliation refuses to run unseeded (ee77a56); crowdin-sync and snode-list move to 13:00 Melbourne, clear of the workflows they replace (b9abc48); the real generators are under test (9419fc5);crowdin-approve-stringsis removed (14cccbe).b31486b); a corrupt state file is a cache miss (4545ae5); a rejected push quotes the server's reason (49e5e2e).a98d264), transcripts use the resolved model id (471dcbd), a failed user lookup no longer fails the digest (ae74899).deploy/README.mdcut from 543 to 331 lines (4cc1c70); stale paths and the apt repository fixed (d4903f9,4bf8093,57f9316).Third review
dc06348); a PR title's links show as their URL, never as masked text (cbc320c).claude-queued, rather than reading as refused and being dropped (da72a72).70618e4); install.sh refuses a tree any account but root can write to (9aed9ef); the Crowdin route's nginx errors, which carry the webhook secret, go to a root-only log (7d2564a); each job's state is private to its account, directories 0711 so the backstop can still read thealertedmarker (68d7306).274e759); reconciliation's run time is the measured 9 minutes, 85 without CroQL (5c8a87b).13bdf16); a relative work directory clones where it names (651cf6c)./etc/session-ops/rehearsalpublishes[Rehearsal]pull requests fromrehearsal/branches, opens a pull request instead of pushing to session-localization'smain, and solves no Zendesk tickets (2dfba56).Not checked live
Decisions worth a look
ghCLI; pushes are plain git.triage.pysplit.20f0e98moves code and nothing else. Each of the 112 top-level definitions keeps its source text, checked by comparing every definition before and after. Against the live Zendesk account, the digest's--dump-batchand the resolver's dry run are byte-identical before and after the whole series. Names are unchanged, so callers differ only in which module they name.GITHUB_PUBLISH_TOKENis gone; withoutGITHUB_APP_IDand its key, crowdin-sync and snode-list exit naming what is missing (188759d).024b4eb). The Zendesk digest keeps its fixed 72 hours.Retry-Afterorx-ratelimit-remaining: 0, GitHub's two rate-limit signals. It waits forx-ratelimit-resetonly in the second case, because GitHub sends that header on every response.