Skip to content

fix: save whole checkpoints in autocommit, without a transaction (TC-2961) - #592

Merged
samgbafa merged 3 commits into
mainfrom
feat/tc-2961-autocommit-save
Nov 13, 2026
Merged

samgbafa merged 3 commits into
mainfrom
feat/tc-2961-autocommit-save

Conversation

@samgbafa

@samgbafa samgbafa commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Linear: TC-2961 (follow-up to #583). Related: TC-2979.

Behaviour

  • No transaction around a checkpoint save. On PostgreSQL, a SQL or DuckDB whole-checkpoint save over 8 MiB no longer opens a transaction.
    • The metadata read (revision, size, checkpoint hash) runs as one autocommit statement on the pool.
    • The compare-and-swap UPDATE, or the primary-key-guarded INSERT for a new database, also runs as one autocommit statement.
    • No transaction, row lock or pinned connection is held across the upload. A cancelled save leaves nothing open.
  • Concurrency guard unchanged. The compare-and-swap still filters on the revision and checkpoint hash the save read. A concurrent writer between the read and the write still fails the save ("concurrent database artifact update"). The Absent case still relies on the primary key.
  • Timeout. The write runs under the base statement_timeout (30 s by default) instead of the 120 s bulk one. Payloads are capped at 64 MiB by fix: refuse SQL checkpoints over 64 MiB and back off failed checkpoint saves (TC-2961) #583's write limit.
  • Outcome of a server timeout. A >8 MiB save that the server cancels at statement_timeout (57014) applied nothing: it is one autocommit statement and is never retried. It now answers a retryable 503, as ≤8 MiB saves and deltas already did. Before, the bulk transaction marked the request possibly applied at BEGIN, so it answered 500 outcome_unknown. A connection lost after the statement was sent, or the SQL service's own transfer bound expiring, is still outcome_unknown.
  • Removed:
    • RecoveringConnection::begin_bulk / begin_bulk_read, BulkScope and StatementTimeouts::bulk. Nothing else used them; hydration already reads in 4 MiB autocommit slices.
    • The database.bulk_statement_timeout_ms setting. DatabaseConfig does not deny unknown fields, so a value still set in a config file or the environment is ignored. The node logs one warning naming it at startup (removed setting is set and ignored). README, tinycloud.toml sample and CHANGELOG updated.

Tests

  • New: tinycloud-node-server/tests/postgres_pool_timeouts.rs::postgres_checkpoint_saves_send_no_begin.
    • A recording TCP proxy sits between the production pool (session statement timeout on) and PostgreSQL.
    • A 9 MiB Absent save and a 9 MiB full-checkpoint save send no BEGIN.
    • A begin() through the same pool is seen, which shows the recorder works.
    • It fails on the previous code, whose bulk scope sent BEGIN for both saves.
  • New, from Bobbie's review (same file):
    • postgres_a_checkpoint_save_after_a_concurrent_delta_is_refused: a full save reads revision 1 on checkpoint A. While its UPDATE waits on the row, a delta-style update moves the row to revision 2 on the same checkpoint. The full save gets the conflict and the delta's row stands. This pins the revision predicate: without it the delta is lost.
    • postgres_a_timed_out_checkpoint_update_is_refused_once_and_not_applied: a 57014 on the >8 MiB UPDATE is refused once (one trigger entry), not tracked as possibly applied, and the row is unchanged. This pins "never retried".
    • These, the no-BEGIN test and postgres_sql_statement_timeout_is_retryable_and_releases_the_lock hold one shared test lock (LARGE_TRANSFERS). Running them at once made the latter's timed refusal flaky on a shared server (3/27 in review).
  • New: removed_settings_tests::a_set_removed_setting_is_named: a configuration that sets database.bulk_statement_timeout_ms is named by the startup check and still loads; a default one is not named.

Existing tests changed (test-ruling: TC-2961)

Removed with the bulk scope, not rewritten:

  • postgres_statement_timeout_modes_scopes_and_migrations: the two bulk-scope blocks (previously at :369 and :423) and the bulk_statement_timeout_ms setting they used. The session, local, off and migration checks are unchanged.
  • postgres_sql_statement_timeout_is_retryable_and_releases_the_lock:
    • The step asserting that a >8 MiB full checkpoint survives a 1.5 s trigger under the bulk timeout is replaced by the new outcome. The same 9 MiB full checkpoint now hits the base timeout and answers Unavailable (503). The lock is released and nothing of it is applied (the next read sees 1 row).
    • The final row count drops from 3 to 2 accordingly, and the bulk_statement_timeout_ms setting is removed.
  • postgres_hydration_slices_run_under_the_base_timeout: removed the bulk_statement_timeout_ms setting (compile only; reads never used it).
  • db_pool::tests::statement_timeouts_render_per_mode: removed the bulk field (compile only).
  • db_pool::tests::track_writes_marks_only_writes_that_may_have_applied: removed the begin_bulk_read step.
  • db_pool::tests::bulk_scope_is_the_pool_off_postgres: removed. Its one non-bulk check (local mode issues nothing off PostgreSQL) is kept as local_mode_issues_nothing_off_postgres.
  • runtime::tests config defaults: removed bulk_statement_timeout_ms from the defaults tuple (compile only).
  • config::tests::startup_rejects_a_statement_timeout_postgres_cannot_hold (from fix: end PostgreSQL transactions left idle for 5 min (TC-2954) #586): removed its bulk_statement_timeout_ms dimension. The statement_timeout_ms bound (INT_MAX accepted, INT_MAX + 1 refused) is unchanged. DatabaseConfig::validate drops the matching bulk bound, and the CHANGELOG entry for fix: end PostgreSQL transactions left idle for 5 min (TC-2954) #586 no longer names the removed setting.

@samgbafa
samgbafa force-pushed the feat/tc-2961-autocommit-save branch from 0572410 to d2f96bf Compare November 13, 2026 20:38
@samgbafa
samgbafa marked this pull request as ready for review November 13, 2026 21:36
@samgbafa
samgbafa merged commit 822ed70 into main Nov 13, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant