Repository navigation
Clear a cached artifact and queue its file in one transaction - #506
Merged
Merged
Conversation
…e transaction, so a failed queue write keeps the record instead of leaving a file nothing points at. Eviction drops the queue entry again once it has deleted the file itself.
andrew
requested changes
Oct 18, 2026
andrew
left a comment
Contributor
There was a problem hiding this comment.
Please handle replacement-file cleanup when the transaction rolls back because the cleanup queue refuses writes, and cover it through the download entrypoint.
…, it is still served to everyone sharing the fetch and deleted once the last of them has closed it, so it no longer stays in storage with nothing pointing at it. A record that committed despite the reported error keeps its file.
…e transaction itself, so that part of this branch is gone; ClearArtifactCache, eviction and the unrecorded refetch remain, and an unrecorded refetch counts as failed for instances waiting on its lease.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #467.
ClearArtifactCache(and soDiscardArtifact) andUpsertArtifactnow write the record and thepending_deletesentry in one transaction, the same wayClearExpiredArtifactdoes since #428. If queueing fails, the record keeps its old path, so the file is never left in storage with nothing pointing at it.Eviction gets this from
ClearArtifactCache: the path is queued before the file is deleted, and the entry is dropped again once the delete worked. A failed delete leaves it queued for reclaim, as before.I ran the build against SQLite and Postgres under load: concurrent fetches, stored files deleted and corrupted underneath the proxy, eviction with a tiny
max_size, andkill -9mid-load. None of the three paths orphaned a file.One trade-off: if only the queue keeps refusing writes while the rest of the database works, a record whose file is missing or corrupt can no longer be cleared or replaced. Every request then refetches from upstream and the new copy is orphaned. In my test with an insert-refusing trigger on
pending_deletes, that left more orphans than main does. A real database failure would hit both writes, so I left it as it is.Not covered here: a crash between writing a fetched file and committing its record in
fetchAndCachestill orphans that file. That happens on main as well; I can open a separate issue for it.