Skip to content

feat: --stats per-file token breakdown; exact post-run token count (#21 P7) - #68

Merged
igorls merged 5 commits into
masterfrom
feat/21-stats
Oct 12, 2026
Merged

igorls merged 5 commits into
masterfrom
feat/21-stats

Conversation

@igorls

@igorls igorls commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Part of #21 (P7). New --stats[=N] flag (require_equals so it never swallows the next argument), render_stats unit-tested in-process, run_with_args stats test, binary tests in tests/test_stats.rs. The post-run 'Estimated tokens' now reads back the written file and counts it with the selected encoding (the >128K warning uses it too); stdout keeps the bytes/4 estimate since the document is gone. Adds a stats field to Args/ResolvedConfig (mechanical edits to Args literals in tests). One existing test fixture (100K+ warning) used a single repeated letter that tokenizes to a quarter of bytes/4; it now uses content with ~1 token per pair.

Copilot AI balanced review requested due to automatic review settings October 11, 2026 15:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Stats can trigger an irrelevant overwrite prompt and can misreport budget boundaries due to divergent header accounting.

2 open findings
What changed in this PR

Adds per-file token statistics and exact post-run token counting.

Changes:

  • Adds --stats[=N] with budget-aware file breakdowns.
  • Uses the selected tokenizer for post-run file counts.
  • Adds unit, integration, documentation, and fixture updates.
File Description
src/​cli.rs Defines --stats.
src/​config_resolver.rs Propagates stats configuration.
src/​lib.rs Renders stats and exact token reports.
tests/​test_stats.rs Tests stats and token reporting.
tests/​test_noninteractive_prompts.rs Updates large-output fixture.
tests/​cli_integration.rs Updates argument fixtures.
tests/​test_auto_diff.rs Updates argument fixtures.
tests/​test_binary_file_autodiff.rs Updates argument fixtures.
tests/​test_comprehensive_edge_cases.rs Updates argument fixtures.
tests/​test_config_resolution.rs Propagates stats in test resolution.
tests/​test_content_filter.rs Updates argument fixtures.
tests/​test_cwd_independence.rs Updates argument fixtures.
tests/​test_determinism.rs Updates argument fixtures.
tests/​test_file_metadata.rs Updates argument fixtures.
tests/​test_inclusion.rs Updates argument fixtures.
tests/​test_parallel_memory.rs Updates argument fixtures.
tests/​test_phase4_integration.rs Updates argument fixtures.
benches/​context_bench.rs Updates benchmark arguments.
README.md Documents --stats.
SKILL.md Adds stats usage guidance.
CHANGELOG.md Records the feature.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/lib.rs
Comment thread src/lib.rs Outdated
@igorls
igorls merged commit 004c793 into master Oct 12, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants