Skip to content

csv-parse: from_line counts non-comment lines when comment is set - #544

Open
smoltafet wants to merge 3 commits into
adaltas:masterfrom
smoltafet:fix/from-line-ignore-comment-lines
Open

smoltafet wants to merge 3 commits into
adaltas:masterfrom
smoltafet:fix/from-line-ignore-comment-lines

Conversation

@smoltafet

@smoltafet smoltafet commented Oct 25, 2026 •

Copy link
Copy Markdown

Fixes #422.

Updated after @wdavidw's feedback, twice over: from_line keeps counting physical lines by default (no breaking change), and the comment-skipping behavior now lives on from_line itself instead of a separate option. from_line takes two forms:

parse(input, { comment: '//', from_line: 3 })
// counts physical lines, as before

parse(input, { comment: '//', from_line: { position: 3, skip_comments: true } })
// starts emitting at the 3rd non-comment line, however many
// comment lines precede or surround it

An integer is normalized to an object with the integer as position and skip_comments set to false; in an object, skip_comments is optional and defaults to false.

Implementation notes:

  • The from_line activation check subtracts the number of full comment lines seen so far only when skip_comments is true. That count is taken when a comment starts (new state.fullCommentLines), not at the record delimiter: while emission is disabled by from_line, __onField short-circuits and the record/field state at the delimiter no longer reflects the line's content, which would misclassify lines with trailing comments as comment lines.
  • A line is a full comment line when nothing but the comment was seen on it: no field characters were appended and no quoted field was opened (tracked as state.lineHasContent). A trailing comment after content does not make the line a comment line. info.comment_lines and error line numbers still use physical lines, unchanged.
  • Normalization validates the object form (position must be a positive integer, skip_comments a boolean); the normalized option is typed in options.d.ts and the API types test covers both forms.
  • This also fixes a pre-existing state leak in the same code path: when the from_line transition (or the skip_empty_lines skip) fired on a line ending with a trailing comment, the early continue skipped the state.commenting = false reset, so the whole next line was silently swallowed as a comment. On master, parse('a,b # comment\n1,2\n3,4', { comment: '#', from_line: 2 }) returns only ['3','4'] and reports comment_lines: 1; it now returns both records and comment_lines: 0. Both branches now reset commenting before continuing.
  • to_line is intentionally left counting physical lines, as in the issue's request; happy to align it too if you prefer.

Tests: eight new tests in packages/csv-parse/test/option.from_line.js covering the object form (leading comment lines using the issue's example, comments between records, boundary positioning, trailing comments, columns: true after comments, validation of the object form, skip_comments unset, and a default-behavior test showing comment lines are counted for a plain integer). Full csv-parse suite: 637 passing, 2 pending (pristine master: 629 passing, 2 pending); tsc --noEmit and eslint are clean. Default behavior was also compared against pristine master on the distinguishing cases and is unchanged.

@smoltafet

Copy link
Copy Markdown
Author

Hi @wdavidw, could you take a look at this when you have a moment? Happy to fix up anything that needs changing. Thanks!

@wdavidw

wdavidw commented Oct 29, 2026

Copy link
Copy Markdown
Member

Hi @smoltafet, I have been receiving a flood of PR recently, alimented by AI, and I don't have the time to review them. I quickly looked at it recently. If I understand your proposition correctly, counting lines, even those containing comments, is on purpose. The idea is that if someone open a document and found the CVS records start in line 200, then parsing shall start there, whichever comes before. Another argument is that the PR introduces a breaking change. What could be done is the introduction of a new option, something like from_line_skip_comments.

@smoltafet

Copy link
Copy Markdown
Author

@wdavidw thanks, that makes sense. I reworked it so from_line counts physical lines by default like before, and the comment skipping is now opt-in through a new from_line_skip_comments option. Tests pass locally (full csv-parse suite, 637 passing). Happy to rename the option if you prefer something else.

@wdavidw

wdavidw commented Oct 29, 2026

Copy link
Copy Markdown
Member

I am currently thinking that from_line could take 2 forms:

  • an object, with 2 properties eg position: number and skip_comments: boolean
  • an integer, normalized to an object with the integer being the position number and skip_comments set to false

what do you think ?

@smoltafet

Copy link
Copy Markdown
Author

@wdavidw good call, I went with the object form. from_line now takes {position, skip_comments}, and an integer gets normalized to that shape with skip_comments: false, so the separate option from the last push is gone. Tests pass locally (full csv-parse suite 637 passing, tsc and eslint clean).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ignore comment lines for the from_line value

2 participants