Skip to content

Consolidate repository packfiles on a timer - #24

Merged
mergify[bot] merged 1 commit into
mainfrom
pack-maintenance
Oct 22, 2026
Merged

mergify[bot] merged 1 commit into
mainfrom
pack-maintenance

Conversation

@littlenano

Copy link
Copy Markdown
Collaborator

Every git fetch that receives objects writes a new packfile, and nothing consolidates them in this workload:

  • the loose-object gc --auto trigger never fires, because objects arrive packed (a mirror here has 0 loose objects);
  • pack consolidation now goes through a multi-pack-index, which a plain git fetch never creates.

On the nixpkgs mirror this accumulated 34,701 packfiles over five months. Each fetch then had to open every pack, so a fetch that found new objects took 4–5 minutes of one core, and one was always running.

A single git repack --geometric=2 -d --write-midx fixes it, and is cheap enough to repeat:

19 packs -> 2 packs, 688 MB -> 652 MB, 10.7 s
single-object lookup 1.6 s -> 0.07 s
fetch 8-13 s (4.5 min when new objects appeared) -> 1 s

This adds services.commit-notifier.maintenance (enabled by default): a oneshot service running that repack over /var/lib/commit-notifier/repositories/*/repo, scheduled by OnCalendar = cfg.maintenance.calendar (default daily) with Persistent and a randomised delay, as the commit-notifier user with Nice = 10 and idle I/O scheduling so it does not compete with the fetches.

Co-authored-by: Lin Yinfeng <lin.yinfeng@outlook.com>
@mergify

mergify Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Queued — the merge queue status continues in this comment ↓.

@mergify

mergify Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Merge Queue Status

This pull request spent 1 minute 16 seconds in the queue, including 32 seconds running CI.

Required conditions to merge
  • #check-failure = 0
  • check-success = check

@mergify mergify Bot added the queued label Oct 22, 2026
@mergify
mergify Bot merged commit bceca5e into main Oct 22, 2026
6 checks passed
@mergify mergify Bot removed the queued label Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant