Repository navigation
fix(files): do not read an ordinary data row as a total row - #9538
Harvey DU (Luckyou-d) wants to merge 3 commits into
Conversation
`total_row` treats the last row as a total when a numeric column's last value equals the sum of the column above it. Any ascending sequence satisfies that, so 1/2/3, 2/4/6 and 10/20/30 lost their last row from `describe`, `value_counts`, `group_by` and the column profile, while `rows` still showed it. A row that carries its own label is data, however neatly its numbers add up: a real total row has no row label, or one of the recognised ones. Both shapes the existing tests cover keep working - `["Total", 6]` through the label branch and `[None, 6]` through the empty first cell. Fixes crewAIInc#9537
Strix Security ReviewWarning This pull request has 28 commits after the last Strix review ( ✅ No open security findings. PR summary: Fixes spreadsheet total-row detection so an ordinary data row whose numbers happen to add up (an ascending sequence such as 1/2/3 or 10/20/30) is no longer mistaken for a total and silently dropped from describe, value_counts, group_by and the overview's column profile; a row counts as a total only when its first cell is blank or a recognised total label. Review summaryThe PR narrows the total-row heuristic in Ruled out (2)
Review 1 on this PR · Reviewed by Strix |
Summary
total_rowtreats the last row as a total when a numeric column's last value equals the sum of the column above it. Any ascending arithmetic sequence satisfies that, so ordinary data silently disappeared fromdescribe,value_counts,group_byand the overview's column profile whilerowsstill showed it:2,4,6and10,20,30behave the same.total_rowis also the function the strix/cursor reviews of #9389 kept landing on.Change
A row that carries its own label is data, however neatly its numbers add up; a real total row has no row label, or one of the recognised ones. Both shapes the existing tests cover keep working -
["Total", 6]through the label branch,[None, 6]through the empty first cell.Verification
pytest lib/crewai-files/tests/spreadsheets/test_answers.py-> 75 passed; the three new sequence cases fail onmainand pass here, and the labelled/unlabelled total-row cases still passpytest lib/crewai-files/tests/spreadsheets/test_reading.py-> 85 passed (160 together withtest_answers)ruff checkandruff format --checkclean on both filesFixes #9537