Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@
- `POST /delegate` and `POST /revoke` check the signature (and, for a renewal lease, the signature of the stored original it renews) before taking any lock. A forged or slow-to-verify request no longer holds the delegation's chain guards, its spaces' write order or a database transaction while it is checked; it is refused at once. A renewal lease whose original is not stored is also refused before any lock. Inside the transaction only the signed time window is checked again. A resubmitted delegation that is already stored is now also signature-checked before it is answered as existing. Details: TC-2986.
- Add opt-in PostgreSQL checkpoint parts for SQL persistence, with explicit artifact/account capacities and streamed hydration. A database stored in parts loads in bounded statements and is not held to the single-row read limit (`storage.sql.max_artifact_read_bytes`); a load that keeps the whole database in memory (a DuckDB export) still waits for one of the large-transfer slots. Details: TC-2917.
- Policy/v3: minting a session and registering a policy no longer re-verify signatures or re-run the claim, presentation, credential admission and allow evaluations while holding their spaces' write order and row locks. Under the locks they check that the challenge, registration and roots are unchanged; those decisions were made before the locks, at the same time, from the same inputs. Refusal codes are unchanged, except that `credential-evaluation-changed` and `policy-evaluation-changed` can no longer occur. Details: TC-2986.
- Artifact-part writers prioritize their own predecessor reclamation, keep replacement contention retryable, and distinguish capacity refusals from space-growth limits; existing database layouts can checkpoint before adopting WAL parts. Details: TC-2917.
- DuckDB saves a complete checkpoint when a write changes its main file, and uses incremental WAL persistence only when that file still matches the stored checkpoint, so acknowledged writes remain recoverable after restart. Details: TC-2990.
- Opt-in SQL artifact parts also stream cumulative WAL updates, retaining the checkpoint until a full checkpoint is needed. Details: TC-2917.
- DuckDB can persist checkpoints, cumulative WAL and imports through the same opt-in bounded artifact parts. Details: TC-2917.
- SQLite stamp replay deadlines now use the same timestamp encoding as cleanup (TC-2920).
Expand Down
23 changes: 22 additions & 1 deletion docs/operations/chunked-sql-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,10 @@ the same database's credit for a second simultaneous replacement.
Published parts have a distinct state and infinite lease, and are excluded from
both global and owner-local reclamation indexes. Inline cleanup is best effort,
scoped to the uploading owner, at most 16 steps and 100 ms per pass; its errors
do not fail the save. Background passes are limited to 128 steps and one second
do not fail the save. A writer first prioritizes its own database's eligible
predecessors for at most two seconds; active upload leases are never stolen.
Failed owners are excluded for the remainder of a collection pass so other
owners can continue. Owner-lock waits at reservation are bounded at 50 ms. Background passes are limited to 128 steps and one second
per 10-second tick. GC lock acquisition waits at most 50 ms per statement.
An owner already above its quota can replace a database with an equal or
smaller image; that exception admits no growth and does not free pending shrink
Expand Down Expand Up @@ -135,3 +138,21 @@ shrink. Size refusal occurs after native execution;
the separate snapshot/headroom acceptance must cover transient oversized
writes and temporary copies before enabling. Operators check CVM disk headroom
against the number and ceilings of open databases, plus that transient cost.

Artifact, account and single-statement capacity refusals return 507. Existing
per-space growth refusals retain 402. A concurrent replacement or contended
reservation returns a retryable 503 without checkpoint back-off; staged
bookkeeping does not turn this positive pre-publication refusal into an
unknown write outcome. A genuinely lost publish reply retains its existing
unknown-outcome behavior. Expired uploads remain counted until collection.

An old-format checkpoint can be replaced by a full owned snapshot before WAL
parts attach. For DuckDB, both legacy and owned-WAL persistence first compare
the local main file's length and CID with the stored checkpoint under the
actor fence. A mismatch requires a complete checkpoint, including blocks
written directly to the main file by a large insert.

The feature remains off. Additional file-removal lock duration, spill-directory
cleanup, import failure cleanup and disk/spill admission conditions still need
their separate acceptance before activation, alongside conversion and the
ceiling-load benchmark.
17 changes: 15 additions & 2 deletions tinycloud-core/src/database_artifacts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ pub enum DatabaseArtifactError {
Db(DbErr),
#[error("artifact storage limit reached")]
StorageLimit,
#[error("database artifact replacement is in progress")]
ReplacementBusy,
#[error("database artifact payload too large: {0} bytes")]
PayloadTooLarge(u64),
/// A payload larger than the repository's single-statement write limit
Expand Down Expand Up @@ -145,7 +147,18 @@ pub enum DatabaseArtifactError {

impl From<DbErr> for DatabaseArtifactError {
fn from(error: DbErr) -> Self {
if error.to_string().contains("TC2917_QUOTA") {
use sea_orm::error::{RuntimeErr, SqlxError};
let replacement_busy = match &error {
DbErr::Conn(RuntimeErr::SqlxError(SqlxError::Database(error)))
| DbErr::Exec(RuntimeErr::SqlxError(SqlxError::Database(error)))
| DbErr::Query(RuntimeErr::SqlxError(SqlxError::Database(error))) => {
error.code().as_deref() == Some("TC001") && error.message() == "TC2917_BUSY"
}
_ => false,
};
if replacement_busy {
Self::ReplacementBusy
} else if error.to_string().contains("TC2917_QUOTA") {
Self::StorageLimit
} else if error.to_string().contains("TC2917_LAYOUT") {
Self::StaleLineage
Expand Down Expand Up @@ -783,7 +796,7 @@ pub struct SeaOrmDatabaseArtifactRepository {
#[cfg(test)]
race_barrier: Option<std::sync::Arc<tokio::sync::Barrier>>,
#[cfg(test)]
chunk_publish_barrier: Option<std::sync::Arc<tokio::sync::Barrier>>,
pub(crate) chunk_publish_barrier: Option<std::sync::Arc<tokio::sync::Barrier>>,
#[cfg(test)]
chunk_reclaim_paused: bool,
}
Expand Down
Loading
Loading