Skip to content

Guard against rules that don't advance state.line / state.pos, as markdown-it JS does #441

Description

@acaprino

Context

Since 13.0.2 (markdown-it/markdown-it#847), markdown-it JS stops with an error when a rule reports a match without consuming input, instead of looping:

  • ParserBlock.tokenize throws block rule didn't increment state.line (parser_block.ts);
  • ParserInline.skipToken and ParserInline.tokenize throw inline rule didn't increment state.pos (parser_inline.ts).

markdown-it-py has no equivalent check in ParserBlock.tokenize or in ParserInline.skipToken / tokenize. A plugin rule that returns True without advancing makes parse() loop forever. It usually pushes a token on every pass, so memory grows until the process is killed:

from markdown_it import MarkdownIt

md = MarkdownIt("commonmark")
md.block.ruler.before("paragraph", "stuck", lambda state, start, end, silent: True)
md.parse("text\n")  # never returns; same with an inline rule that returns True

This is not hypothetical. executablebooks/mdit-py-plugins#156 (texmath, input "> $$ a=1\n\n$$\n") and, earlier, executablebooks/mdit-py-plugins#117 (amsmath) were both this bug in a plugin. In each case the only symptom was a hang.

Proposal

Port the JS guards: after a rule returns True, raise if state.line (block) or state.pos (inline) has not advanced. A buggy plugin would then fail fast with a clear message that names the problem, instead of hanging and exhausting memory. Built-in rules always advance, so this only affects plugins that are already broken.

Reproduced on markdown-it-py 4.2.0 and master (6f58654), Python 3.12. I'm happy to open a PR if you'd like one.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions