Skip to content

feat: report both ratios every --report-interval seconds of trace time, as cachesim does - #5

Merged
LauYeeYu merged 3 commits into
mainfrom
feat/report-interval
Nov 10, 2026
Merged

LauYeeYu merged 3 commits into
mainfrom
feat/report-interval

Conversation

@LauYeeYu

Copy link
Copy Markdown
Collaborator

@haochengxia asked on #4 (comment, follow-up) for interval-level reporting like libCacheSim's: both metrics every N requests or over fixed trace-time windows, to see how cache performance changes through a trace. Until now prefixsim printed only the totals at the end of each run. This adds the trace-time version, modeled on cachesim's --report-interval.

What this does

--report-interval <s> prints a line every s seconds of trace time with the block hit ratio and compute saving ratio, both over the run so far and over the interval just closed:

freeinference.lcp.zst lru 24.00 hour: 15761 requests, block hit ratio 0.9439, interval block hit ratio 0.9439, compute saving ratio 0.9391, interval compute saving ratio 0.9391
freeinference.lcp.zst lru 48.00 hour: 33679 requests, block hit ratio 0.9434, interval block hit ratio 0.9428, compute saving ratio 0.9390, interval compute saving ratio 0.9389
...
freeinference.lcp.zst lru 168.00 hour: 327548 requests, block hit ratio 0.9175, interval block hit ratio 0.8967, compute saving ratio 0.9259, interval compute saving ratio 0.9118

It follows cachesim (libCacheSim/bin/cachesim/sim.c):

  • Same option. It has the same name, takes seconds of trace time, and measures from the first request.
  • Same rule for closing an interval. An interval closes at the first request at least s seconds after the previous report, and that request belongs to the interval it closes.
  • Same wording. cachesim's <hours> hour: <n> requests, miss ratio X, interval miss ratio Y becomes the two prefixsim ratios, so a script like scripts/plot_mrc_time.py needs only a changed regex.

Where it differs from cachesim:

  • Off by default. When cachesim runs a single cache, it reports every day of trace time by default. Here a run without the option prints exactly what it printed before.
  • stdout, not stderr. cachesim writes these lines as INFO logs to stderr. Here they go to stdout, flushed as they happen, next to the summary and RESULT lines.
  • A last report at the end of the trace. cachesim stops at its last full interval. Here, whatever was served after the last report gets one more line, so the intervals always add up to the RESULT line.
  • Skipped requests are left out. Requests larger than the cache are excluded from every count, as they are from every other statistic, and they do not close an interval.

Inside the simulator, SimulatorConfig gains report_interval_sec and an on_report callback that receives an IntervalReport with the totals so far and the interval's own Stats. The figures are read off phase 1's statistics, so the simulation itself is unchanged.

Reporting every N requests instead would be a small addition on top of this, if it turns out to be useful.

How it was tested

  • testPrefixsim and testLcp pass, also under LeakSanitizer.
  • New test. In a hand-traceable 11-request trace with one skipped request, reports close at the expected times (3, 6, 9 and 10 s) and cover the expected requests. The intervals add up exactly to the final Stats, the last report's totals equal them, and running without reports gives the same result and no report.
  • Streaming lcp path. On the full FreeInference trace, LRU and S3-FIFO at 64k blocks under qwen3coder30b_blksz_16: with --report-interval 86400 the RESULT lines are identical to a run without the option. The example above is from that run.
  • Load-whole JSONL path. On the first 20k FreeInference requests as JSONL, LRU at 16k blocks: with --report-interval 3600 the RESULT line is again identical.

@LauYeeYu
LauYeeYu merged commit 160d5f8 into main Nov 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant