diff --git a/CHANGELOG.md b/CHANGELOG.md index a642328b..0d4785c2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ - Add opt-in PostgreSQL checkpoint parts for SQL persistence, with explicit artifact/account capacities and streamed hydration. A database stored in parts loads in bounded statements and is not held to the single-row read limit (`storage.sql.max_artifact_read_bytes`); a load that keeps the whole database in memory (a DuckDB export) still waits for one of the large-transfer slots. Details: TC-2917. - Policy/v3: minting a session and registering a policy no longer re-verify signatures or re-run the claim, presentation, credential admission and allow evaluations while holding their spaces' write order and row locks. Under the locks they check that the challenge, registration and roots are unchanged; those decisions were made before the locks, at the same time, from the same inputs. Refusal codes are unchanged, except that `credential-evaluation-changed` and `policy-evaluation-changed` can no longer occur. Details: TC-2986. - Opt-in SQL artifact parts also stream cumulative WAL updates, retaining the checkpoint until a full checkpoint is needed. Details: TC-2917. +- DuckDB can persist checkpoints, cumulative WAL and imports through the same opt-in bounded artifact parts. Details: TC-2917. - SQLite stamp replay deadlines now use the same timestamp encoding as cleanup (TC-2920). - PostgreSQL database artifacts use uncompressed external storage for new writes to avoid repeated decompression during chunked reads (TC-2917). Existing large artifacts require an operator re-toast; verify both storage settings after rollout and follow `scripts/tc2917-README.md` if lock contention deferred the migration. - Encryption: v2 column format with key ids, per-purpose keys and recorded format metadata. Optional strict reads are off by default. Details: TC-2516, TC-2517. diff --git a/docs/operations/chunked-sql-artifacts.md b/docs/operations/chunked-sql-artifacts.md index f6abedf4..65aa1651 100644 --- a/docs/operations/chunked-sql-artifacts.md +++ b/docs/operations/chunked-sql-artifacts.md @@ -1,6 +1,6 @@ -# Opt-in SQL checkpoint parts +# Opt-in database artifact parts -Opt-in SQL persistence stores checkpoints and cumulative WAL as immutable PostgreSQL parts. +Opt-in SQL and DuckDB persistence stores checkpoints and cumulative WAL as immutable PostgreSQL parts. Each payload statement carries at most 1 MiB. A metadata-only compare-and-swap publishes a sealed part; successful writes retain the existing durability and lineage semantics. Hydration verifies the part CID before publishing its cache. @@ -38,13 +38,15 @@ bytes are gone. A periodic collector recovers uploads whose 120-second lease expired. Lease expiry determines eligibility; part identity and state fence uploads against concurrent reclamation. Active sealed uploads retain their lease until publication. Deleted artifacts are reclaimed by the same mechanism. -It does not enable chunked DuckDB writes. Legacy data remains readable. Drain +DuckDB uses owned checkpoint and WAL files under the same capacities. Its +imports stream the already-buffered public request into bounded artifact +statements. Legacy data remains readable. Drain older writers before enabling this format; turning the switch off does not convert published parts back to legacy blobs. Current readers understand both layouts; legacy persistence refuses a database already using parts. Do not enable for prolonged operation until the approved ceiling load acceptance and -remaining conversion and DuckDB slices are complete. On each test release, report +remaining conversion slice is complete. On each test release, report reserved, staged and retained bytes from the following metadata-only inventory: ```sql @@ -118,3 +120,18 @@ Roll out WAL-capable readers to every instance before enabling the switch. Checkpoint-only part readers fail closed on WAL layouts. Schema changes are forward-only; switching the feature off keeps existing layouts readable and writable by the current binary, and does not downgrade their format. + +New DuckDB databases with parts enabled start file-backed under the existing +node data directory, regardless of the in-memory threshold. Their actor fence +protects checkpoint/WAL copies. Closing or idle-evicting an actor removes its +cache files after the native connection is closed; the next access rehydrates +from durable parts. Legacy actors retain their existing storage behavior. + +For disk headroom, each successfully persisted open database retains at most +the configured artifact ceiling in checkpoint plus cumulative WAL. A full +checkpoint also needs an owned temporary copy while streaming, and old/new +versions overlap in PostgreSQL until their deletions commit. PostgreSQL can reuse deleted storage after vacuum; relation files need not +shrink. Size refusal occurs after native execution; +the separate snapshot/headroom acceptance must cover transient oversized +writes and temporary copies before enabling. Operators check CVM disk headroom +against the number and ceilings of open databases, plus that transient cost. diff --git a/tinycloud-core/src/duckdb/database.rs b/tinycloud-core/src/duckdb/database.rs index 824869b8..f5ed1864 100644 --- a/tinycloud-core/src/duckdb/database.rs +++ b/tinycloud-core/src/duckdb/database.rs @@ -35,6 +35,13 @@ enum DbMessage { Export { response_tx: oneshot::Sender, DuckDbError>>, }, + Snapshot { + response_tx: oneshot::Sender>, + }, + WalSnapshot { + max_bytes: u64, + response_tx: oneshot::Sender, DuckDbError>>, + }, Wal { response_tx: oneshot::Sender>, DuckDbError>>, }, @@ -96,6 +103,40 @@ impl DatabaseHandle { .map_err(|_| DuckDbError::Internal("Database actor dropped response".to_string()))? } + pub async fn snapshot(&self) -> Result { + let (response_tx, response_rx) = oneshot::channel(); + self.tx + .send(( + request_telemetry::current(), + DbMessage::Snapshot { response_tx }, + )) + .await + .map_err(|_| DuckDbError::Internal("Database actor not available".into()))?; + response_rx + .await + .map_err(|_| DuckDbError::Internal("Database actor dropped response".into()))? + } + + pub async fn wal_snapshot( + &self, + max_bytes: u64, + ) -> Result, DuckDbError> { + let (response_tx, response_rx) = oneshot::channel(); + self.tx + .send(( + request_telemetry::current(), + DbMessage::WalSnapshot { + max_bytes, + response_tx, + }, + )) + .await + .map_err(|_| DuckDbError::Internal("Database actor not available".into()))?; + response_rx + .await + .map_err(|_| DuckDbError::Internal("Database actor dropped response".into()))? + } + pub async fn wal(&self) -> Result>, DuckDbError> { let (response_tx, response_rx) = oneshot::channel(); self.tx @@ -131,6 +172,29 @@ pub fn spawn_actor( idle_timeout_secs: u64, max_memory_per_connection: String, databases: Arc>, +) -> DatabaseHandle { + spawn_actor_with_parts( + space_id, + db_name, + base_path, + memory_threshold, + false, + idle_timeout_secs, + max_memory_per_connection, + databases, + ) +} + +#[allow(clippy::too_many_arguments)] +pub(crate) fn spawn_actor_with_parts( + space_id: String, + db_name: String, + base_path: String, + memory_threshold: u64, + force_file: bool, + idle_timeout_secs: u64, + max_memory_per_connection: String, + databases: Arc>, ) -> DatabaseHandle { let (tx, mut rx) = mpsc::channel::<(Option>, DbMessage)>(32); let id = NEXT_ACTOR_ID.fetch_add(1, Ordering::Relaxed); @@ -143,8 +207,8 @@ pub fn spawn_actor( .join(&space_id) .join(format!("{}.duckdb", db_name)); - // Check if file already exists -- if so, open from file - let mut mode = if file_path.exists() { + // Part-backed actors start file-backed; legacy actors reopen existing caches. + let mut mode = if force_file || file_path.exists() { StorageMode::File(file_path.clone()) } else { StorageMode::InMemory @@ -175,6 +239,18 @@ pub fn spawn_actor( e )))); } + DbMessage::Snapshot { response_tx } => { + let _ = response_tx.send(Err(DuckDbError::Internal(format!( + "Failed to open: {}", + e + )))); + } + DbMessage::WalSnapshot { response_tx, .. } => { + let _ = response_tx.send(Err(DuckDbError::Internal(format!( + "Failed to open: {}", + e + )))); + } DbMessage::Wal { response_tx } => { let _ = response_tx.send(Err(DuckDbError::Internal(format!( "Failed to open: {}", @@ -186,7 +262,7 @@ pub fn spawn_actor( } } } - databases.remove_if(&(space_id, db_name), |_, handle| handle.id == id); + deregister_actor(&databases, (space_id, db_name), id, &file_path, force_file); return; } } @@ -253,6 +329,18 @@ pub fn spawn_actor( handle_export(&conn, &mode, &file_path, &max_memory_per_connection); let _ = response_tx.send(result); } + DbMessage::Snapshot { response_tx } => { + let result = + handle_snapshot(&conn, &mode, &file_path, &max_memory_per_connection); + let _ = response_tx.send(result); + } + DbMessage::WalSnapshot { + max_bytes, + response_tx, + } => { + let result = handle_wal_snapshot(&mode, &file_path, max_bytes); + let _ = response_tx.send(result); + } DbMessage::Wal { response_tx } => { let result = handle_wal(&mode, &file_path); let _ = response_tx.send(result); @@ -272,9 +360,7 @@ pub fn spawn_actor( // been hydrated and registered for this key, and evicting it would send // the next request through hydration while it is live — deleting and // rewriting the files it holds open by path. - databases.remove_if(&(space_id.clone(), db_name.clone()), |_, handle| { - handle.id == id - }); + deregister_actor(&databases, (space_id, db_name), id, &file_path, force_file); tracing::debug!("Database actor shutting down"); if let Some(response_tx) = shutdown { let _ = response_tx.send(()); @@ -284,6 +370,112 @@ pub fn spawn_actor( DatabaseHandle { id, tx } } +/// Keep registration until cleanup is complete. A replacement actor's cache +/// must never be removed by its predecessor. Explicit shutdown already holds +/// the service's per-database operation fence, even if it removed the entry. +fn deregister_actor( + databases: &DashMap<(String, String), DatabaseHandle>, + key: (String, String), + id: u64, + path: &Path, + remove_files: bool, +) { + use dashmap::mapref::entry::Entry; + let entry = databases.entry(key); + if matches!(&entry, Entry::Occupied(entry) if entry.get().id != id) { + return; + } + if remove_files { + let mut failed = false; + for candidate in [ + path.to_path_buf(), + PathBuf::from(format!("{}.wal", path.display())), + PathBuf::from(format!("{}.tmp", path.display())), + ] { + let result = if candidate.is_dir() { + std::fs::remove_dir_all(&candidate) + } else { + std::fs::remove_file(&candidate) + }; + if let Err(error) = result { + failed |= error.kind() != std::io::ErrorKind::NotFound; + } + } + if failed { + // Operators use this value-free event when checking cache-disk headroom. + tracing::warn!( + event = "duckdb_cache_cleanup_failed", + "Database cache cleanup failed" + ); + } + } + if let Entry::Occupied(entry) = entry { + entry.remove(); + } +} + +fn handle_wal_snapshot( + mode: &StorageMode, + file_path: &Path, + max_bytes: u64, +) -> Result, DuckDbError> { + if !matches!(mode, StorageMode::File(_)) { + return Ok(None); + } + let mut source = match std::fs::File::open(format!("{}.wal", file_path.display())) { + Ok(file) => file, + Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(error) => return Err(DuckDbError::Internal(error.to_string())), + }; + let size = source + .metadata() + .map_err(|e| DuckDbError::Internal(e.to_string()))? + .len(); + if size == 0 || size >= max_bytes { + return Ok(None); + } + let mut snapshot = + tempfile::NamedTempFile::new().map_err(|e| DuckDbError::Internal(e.to_string()))?; + std::io::copy(&mut source, snapshot.as_file_mut()) + .map_err(|e| DuckDbError::Internal(e.to_string()))?; + Ok(Some(snapshot)) +} + +fn copy_snapshot(path: &Path) -> Result { + let mut snapshot = + tempfile::NamedTempFile::new().map_err(|e| DuckDbError::Internal(e.to_string()))?; + let mut source = std::fs::File::open(path).map_err(|e| DuckDbError::Internal(e.to_string()))?; + std::io::copy(&mut source, snapshot.as_file_mut()) + .map_err(|e| DuckDbError::Internal(e.to_string()))?; + Ok(snapshot) +} + +fn handle_snapshot( + conn: &duckdb::Connection, + mode: &StorageMode, + file_path: &Path, + max_memory: &str, +) -> Result { + match mode { + StorageMode::File(_) => { + { + let _timer = QueryTimer::new(); + conn.execute_batch("CHECKPOINT;") + .map_err(|e| DuckDbError::Internal(e.to_string()))?; + } + copy_snapshot(file_path) + } + StorageMode::InMemory => { + let dir = tempfile::tempdir().map_err(|e| DuckDbError::Internal(e.to_string()))?; + let path = dir.path().join("snapshot.duckdb"); + let dest = storage::open_connection(Some(&path), max_memory)?; + storage::copy_tables(conn, &dest)?; + drop(dest); + copy_snapshot(&path) + } + } +} + fn handle_wal(mode: &StorageMode, file_path: &Path) -> Result>, DuckDbError> { if !matches!(mode, StorageMode::File(_)) { return Ok(None); diff --git a/tinycloud-core/src/duckdb/service.rs b/tinycloud-core/src/duckdb/service.rs index 909892e3..0e16e41b 100644 --- a/tinycloud-core/src/duckdb/service.rs +++ b/tinycloud-core/src/duckdb/service.rs @@ -16,7 +16,7 @@ use crate::request_telemetry::{self, QueryTimer}; use super::{ caveats::DuckDbCaveats, - database::{spawn_actor, DatabaseHandle}, + database::{spawn_actor_with_parts, DatabaseHandle}, storage, types::*, }; @@ -318,32 +318,50 @@ impl DuckDbService { return Err(e); } - // Rename temp to final + // Close the previous actor before replacing its files: connection + // shutdown may checkpoint its old WAL into that path. + let key = (space.to_string(), db_name.to_string()); + if let Some((_, handle)) = self.databases.remove(&key) { + handle.shutdown().await?; + } + self.lineage.remove(&key); tokio::fs::rename(&temp_path, &final_path) .await .map_err(|e| DuckDbError::ImportError(format!("Failed to finalize import: {}", e)))?; - - // Remove the existing handle so the next access reopens the database from the new file - let key = (space.to_string(), db_name.to_string()); - self.databases.remove(&key); - // An import deliberately replaces content it never read, so it carries - // no lineage. Drop the evicted actor's record with it: the next access - // rehydrates and records what the import committed. - self.lineage.remove(&key); - - if let Err(e) = self + let persisted = if self .artifact_repository - .save( - "duckdb", - &space.to_string(), - db_name, - data.to_vec(), - ArtifactExpectation::Any, - ) - .await + .chunked_checkpoints("duckdb", &key.0, db_name) { + let snapshot = + tempfile::NamedTempFile::new().map_err(|e| DuckDbError::Internal(e.to_string()))?; + tokio::fs::write(snapshot.path(), data) + .await + .map_err(|e| DuckDbError::ImportError(e.to_string()))?; + self.artifact_repository + .save_snapshot( + "duckdb", + &key.0, + db_name, + snapshot, + ArtifactExpectation::Any, + ) + .await + .map(|_| ()) + } else { + self.artifact_repository + .save( + "duckdb", + &key.0, + db_name, + data.to_vec(), + ArtifactExpectation::Any, + ) + .await + .map(|_| ()) + }; + if let Err(error) = persisted { self.discard_local_state(&key).await?; - return Err(artifact_error_to_duckdb(e)); + return Err(artifact_error_to_duckdb(error)); } Ok(()) @@ -389,11 +407,16 @@ impl DuckDbService { .databases .entry(key) .or_insert_with(|| { - spawn_actor( + spawn_actor_with_parts( space.to_string(), db_name.to_string(), self.base_path.clone(), self.memory_threshold, + self.artifact_repository.chunked_checkpoints( + "duckdb", + &space.to_string(), + db_name, + ), self.idle_timeout_secs, self.max_memory_per_connection.clone(), self.databases.clone(), @@ -522,6 +545,30 @@ impl DuckDbService { let key = (space.to_string(), db_name.to_string()); let expected = self.expectation(&key); + if self + .artifact_repository + .chunked_checkpoints("duckdb", &key.0, db_name) + { + if expected != ArtifactExpectation::Absent { + if let Some(wal) = handle.wal_snapshot(MAX_WAL_DELTA_BYTES as u64).await? { + let saved = self + .artifact_repository + .save_wal_snapshot("duckdb", &key.0, db_name, wal, expected) + .await + .map_err(artifact_error_to_duckdb)?; + self.lineage.insert( + key, + ArtifactExpectation::Derived { + revision: saved.revision, + checkpoint_content_hash: saved.checkpoint_content_hash, + }, + ); + return Ok(()); + } + } + return self.checkpoint(space, db_name, handle, false).await; + } + if let Some(wal) = handle .wal() .await? @@ -567,6 +614,41 @@ impl DuckDbService { ) -> Result<(), DuckDbError> { let key = (space.to_string(), db_name.to_string()); let expected = self.expectation(&key); + if self + .artifact_repository + .chunked_checkpoints("duckdb", &key.0, db_name) + { + let snapshot = handle.snapshot().await?; + if without_growth { + let size = snapshot + .as_file() + .metadata() + .map_err(|e| DuckDbError::Internal(e.to_string()))? + .len(); + let durable = self + .artifact_repository + .load_meta("duckdb", &key.0, db_name) + .await + .map_err(artifact_error_to_duckdb)? + .map_or(0, |m| m.size_bytes.max(0) as u64); + if size > durable { + return Err(DuckDbError::StorageWouldGrow); + } + } + let saved = self + .artifact_repository + .save_snapshot("duckdb", &key.0, db_name, snapshot, expected) + .await + .map_err(artifact_error_to_duckdb)?; + self.lineage.insert( + key, + ArtifactExpectation::Derived { + revision: saved.revision, + checkpoint_content_hash: saved.checkpoint_content_hash, + }, + ); + return Ok(()); + } let payload = handle.export().await?; let bytes = payload.len(); if without_growth { @@ -1409,4 +1491,337 @@ mod tests { other => panic!("expected schema response, got {other:?}"), } } + #[tokio::test] + async fn postgres_chunked_duckdb_save_wal_restart_and_import() { + use sea_orm::{ConnectionTrait, DbBackend, Statement}; + for memory_threshold in [0, u64::MAX] { + let Some(fixture) = crate::test_support::PostgresTestSchema::new( + "postgres_chunked_duckdb_save_wal_restart_and_import", + ) + .await + else { + return; + }; + let repo = Arc::new( + SeaOrmDatabaseArtifactRepository::new(fixture.conn.clone()) + .with_chunk_limits(Some(crate::database_artifacts::chunks::ChunkLimits { + artifact_bytes: 64 * 1024 * 1024, + account_bytes: 128 * 1024 * 1024, + })) + .with_max_write_bytes(1024 * 1024), + ); + let first = TempDir::new().unwrap(); + let second = TempDir::new().unwrap(); + let space = test_space_id("chunked-duckdb"); + let service = DuckDbService::new( + first.path().to_string_lossy().into_owned(), + memory_threshold, + 300, + "128MB".into(), + repo.clone(), + ); + service + .execute( + &space, + "analytics", + DuckDbRequest::Execute { + schema: Some(vec!["CREATE TABLE events(id BIGINT, body VARCHAR)".into()]), + sql: "INSERT INTO events SELECT i, md5(i::VARCHAR) FROM range(100000) t(i)" + .into(), + params: vec![], + }, + None, + "tinycloud.duckdb/admin".into(), + false, + ) + .await + .unwrap(); + let initial = repo + .load_meta("duckdb", &space.to_string(), "analytics") + .await + .unwrap() + .unwrap(); + assert!( + initial.checkpoint_size_bytes > 1024 * 1024, + "test must cross a chunk boundary" + ); + let checkpoint: String = fixture + .conn + .query_one(Statement::from_string( + DbBackend::Postgres, + "SELECT checkpoint_part_id FROM database_artifact".to_string(), + )) + .await + .unwrap() + .unwrap() + .try_get("", "checkpoint_part_id") + .unwrap(); + assert!( + service.cache_path(&space, "analytics").exists(), + "chunked actors start file-backed" + ); + service + .execute( + &space, + "analytics", + DuckDbRequest::Execute { + schema: None, + sql: "INSERT INTO events VALUES (100000, 'last')".into(), + params: vec![], + }, + None, + "tinycloud.duckdb/write".into(), + false, + ) + .await + .unwrap(); + let saved = repo + .load_meta("duckdb", &space.to_string(), "analytics") + .await + .unwrap() + .unwrap(); + assert_eq!(saved.revision, 2); + assert!(saved.delta_size_bytes > 0); + assert_eq!( + saved.checkpoint_content_hash, + initial.checkpoint_content_hash + ); + service + .discard_local_state(&(space.to_string(), "analytics".into())) + .await + .unwrap(); + drop(service); + // A new repository with the flag off still opens published parts. + let off = Arc::new( + SeaOrmDatabaseArtifactRepository::new(fixture.conn.clone()) + .with_max_write_bytes(1024 * 1024), + ); + let reopened = DuckDbService::new( + second.path().to_string_lossy().into_owned(), + 0, + 300, + "128MB".into(), + off, + ); + let result = reopened + .execute( + &space, + "analytics", + DuckDbRequest::Query { + sql: "SELECT count(*),max(CASE WHEN id=100000 THEN body END) FROM events" + .into(), + params: vec![], + }, + None, + "tinycloud.duckdb/read".into(), + false, + ) + .await + .unwrap(); + let DuckDbResponse::Query(query) = result.response else { + panic!("query required"); + }; + assert_eq!( + query.rows, + vec![vec![ + DuckDbValue::Integer(100001), + DuckDbValue::Text("last".into()) + ]] + ); + let handle = reopened + .databases + .get(&(space.to_string(), "analytics".into())) + .unwrap() + .clone(); + reopened + .checkpoint(&space, "analytics", &handle, false) + .await + .unwrap(); + repo.reap_unused_parts(128).await.unwrap(); + let n: i64 = fixture + .conn + .query_one(Statement::from_sql_and_values( + DbBackend::Postgres, + "SELECT count(*)::bigint AS n FROM artifact_chunk WHERE part_id=$1", + [checkpoint.into()], + )) + .await + .unwrap() + .unwrap() + .try_get("", "n") + .unwrap(); + assert_eq!(n, 0, "superseded checkpoint chunks must be gone"); + let exported = reopened.export(&space, "analytics").await.unwrap(); + assert!(exported.len() > 1024 * 1024); + // Import replaces a live actor and uses bounded payload statements. + reopened + .import_db(&space, "analytics", &exported) + .await + .unwrap(); + let imported = repo + .load_meta("duckdb", &space.to_string(), "analytics") + .await + .unwrap() + .unwrap(); + assert!(!imported.has_delta()); + let result = reopened + .execute( + &space, + "analytics", + DuckDbRequest::Query { + sql: "SELECT count(*) FROM events".into(), + params: vec![], + }, + None, + "tinycloud.duckdb/read".into(), + false, + ) + .await + .unwrap(); + let DuckDbResponse::Query(query) = result.response else { + panic!("query required"); + }; + assert_eq!(query.rows, vec![vec![DuckDbValue::Integer(100001)]]); + reopened + .discard_local_state(&(space.to_string(), "analytics".into())) + .await + .unwrap(); + repo.reap_unused_parts(128).await.unwrap(); + let state=fixture.conn.query_one(Statement::from_string(DbBackend::Postgres, + "SELECT max(octet_length(payload))::bigint AS max_bytes,count(*) FILTER(WHERE pg_column_compression(payload) IS NOT NULL)::bigint AS compressed FROM artifact_chunk".to_string())).await.unwrap().unwrap(); + assert!(state.try_get::("", "max_bytes").unwrap() <= 1024 * 1024); + assert_eq!(state.try_get::("", "compressed").unwrap(), 0); + } + } + + #[tokio::test] + async fn postgres_chunked_duckdb_capacity_refusal_discards_local_write() { + let Some(fixture) = crate::test_support::PostgresTestSchema::new( + "postgres_chunked_duckdb_capacity_refusal_discards_local_write", + ) + .await + else { + return; + }; + let repo = Arc::new( + SeaOrmDatabaseArtifactRepository::new(fixture.conn.clone()).with_chunk_limits(Some( + crate::database_artifacts::chunks::ChunkLimits { + artifact_bytes: 1, + account_bytes: 1, + }, + )), + ); + let cache = TempDir::new().unwrap(); + let service = file_service(&cache, repo.clone()); + let space = test_space_id("chunked-duckdb-cap"); + let outcome = service + .execute( + &space, + "main", + DuckDbRequest::Execute { + schema: Some(vec!["CREATE TABLE items(id INTEGER)".into()]), + sql: "INSERT INTO items VALUES (1)".into(), + params: vec![], + }, + None, + "tinycloud.duckdb/admin".into(), + false, + ) + .await; + assert!( + matches!(outcome, Err(DuckDbError::StorageWouldGrow)), + "{outcome:?}" + ); + assert!(repo + .load_meta("duckdb", &space.to_string(), "main") + .await + .unwrap() + .is_none()); + assert!(!service + .databases + .contains_key(&(space.to_string(), "main".into()))); + assert!(!service.cache_path(&space, "main").exists()); + } + #[tokio::test] + async fn postgres_chunked_duckdb_actor_close_and_idle_evict_remove_files() { + let Some(fixture) = crate::test_support::PostgresTestSchema::new( + "postgres_chunked_duckdb_actor_close_and_idle_evict_remove_files", + ) + .await + else { + return; + }; + let repo = Arc::new( + SeaOrmDatabaseArtifactRepository::new(fixture.conn.clone()).with_chunk_limits(Some( + crate::database_artifacts::chunks::ChunkLimits { + artifact_bytes: 4 * 1024 * 1024, + account_bytes: 16 * 1024 * 1024, + }, + )), + ); + let cache = TempDir::new().unwrap(); + let service = DuckDbService::new( + cache.path().to_string_lossy().into_owned(), + u64::MAX, + 1, + "128MB".into(), + repo, + ); + let space = test_space_id("chunked-duckdb-eviction"); + let key = (space.to_string(), "main".to_string()); + service + .execute( + &space, + "main", + DuckDbRequest::Execute { + schema: Some(vec!["CREATE TABLE items(id INTEGER)".into()]), + sql: "INSERT INTO items VALUES (1)".into(), + params: vec![], + }, + None, + "tinycloud.duckdb/admin".into(), + false, + ) + .await + .unwrap(); + let path = service.cache_path(&space, "main"); + assert!(path.exists()); + let handle = service.databases.get(&key).unwrap().clone(); + handle.shutdown().await.unwrap(); + assert!(!path.exists(), "actor shutdown itself removes the cache"); + assert!(!duckdb_wal_path(&path).exists()); + let query = || DuckDbRequest::Query { + sql: "SELECT id FROM items".into(), + params: vec![], + }; + for _ in 0..2 { + let result = service + .execute( + &space, + "main", + query(), + None, + "tinycloud.duckdb/read".into(), + false, + ) + .await + .unwrap(); + let DuckDbResponse::Query(query) = result.response else { + panic!("query required"); + }; + assert_eq!(query.rows, vec![vec![DuckDbValue::Integer(1)]]); + assert!(path.exists()); + let until = tokio::time::Instant::now() + std::time::Duration::from_secs(5); + while service.databases.contains_key(&key) { + assert!(tokio::time::Instant::now() < until, "actor must idle-evict"); + tokio::time::sleep(std::time::Duration::from_millis(10)).await; + } + assert!( + !path.exists(), + "idle eviction removes the closed actor's file" + ); + assert!(!duckdb_wal_path(&path).exists()); + assert!(!PathBuf::from(format!("{}.tmp", path.display())).exists()); + } + } }