From 48132940625b72d22038d49f2ab3d7986229e38c Mon Sep 17 00:00:00 2001 From: "Peter B. Johnson" Date: Sun, 20 Sep 2026 23:38:41 +0100 Subject: [PATCH 1/3] implement: Let a spec reach the items nested inside a block (t51) --- CHANGELOG.md | 1 + docs/source/drafts.md | 11 +- docs/source/spec.md | 39 +++- in2lambda/draft/__init__.py | 33 +++- in2lambda/draft/report.py | 14 ++ in2lambda/source/__init__.py | 173 +++++++++++++++--- in2lambda/spec/__init__.py | 48 +++-- tests/conftest.py | 10 +- tests/fixtures/drafts/README.md | 6 +- tests/fixtures/drafts/nested_list/report.json | 16 -- tests/fixtures/sources/README.md | 9 + .../sources/display_maths/expected.json | 42 +++++ .../fixtures/sources/fenced_div/expected.json | 48 +++++ tests/fixtures/sources/fenced_div/source.tex | 16 ++ .../sources/nested_list/expected.json | 62 +++++++ tests/fixtures/sources/nested_list/source.md | 14 ++ tests/fixtures/specs/README.md | 12 +- .../part_part_sol_sol_nested/expected.json | 98 ++++++++++ .../specs/part_part_sol_sol_nested/spec.yaml | 7 + .../part_part_sol_sol_nested/uncovered.txt | 0 .../part_sol_part_sol_nested/expected.json | 98 ++++++++++ .../specs/part_sol_part_sol_nested/spec.yaml | 5 + .../part_sol_part_sol_nested/uncovered.txt | 0 .../specs/parts_sep_sol_nested/expected.json | 86 +++++++++ .../specs/parts_sep_sol_nested/spec.yaml | 6 + .../specs/parts_sep_sol_nested/uncovered.txt | 0 tests/test_source.py | 32 +++- tests/test_spec.py | 16 +- 28 files changed, 822 insertions(+), 80 deletions(-) create mode 100644 tests/fixtures/sources/fenced_div/expected.json create mode 100644 tests/fixtures/sources/fenced_div/source.tex create mode 100644 tests/fixtures/sources/nested_list/expected.json create mode 100644 tests/fixtures/sources/nested_list/source.md create mode 100644 tests/fixtures/specs/part_part_sol_sol_nested/expected.json create mode 100644 tests/fixtures/specs/part_part_sol_sol_nested/spec.yaml create mode 100644 tests/fixtures/specs/part_part_sol_sol_nested/uncovered.txt create mode 100644 tests/fixtures/specs/part_sol_part_sol_nested/expected.json create mode 100644 tests/fixtures/specs/part_sol_part_sol_nested/spec.yaml create mode 100644 tests/fixtures/specs/part_sol_part_sol_nested/uncovered.txt create mode 100644 tests/fixtures/specs/parts_sep_sol_nested/expected.json create mode 100644 tests/fixtures/specs/parts_sep_sol_nested/spec.yaml create mode 100644 tests/fixtures/specs/parts_sep_sol_nested/uncovered.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 85f7cc3..d72d41d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,5 +17,6 @@ - An export now names its images as they sit in `media/`: every markdown image reference a question holds - in its text, a part's, a worked solution, a final answer or an answer box's wording - is rewritten to the file name the image was carried under, so a document writing `![](figures/train.png)` exports as `![](train.png)` beside `media/train.png` and Lambda Feedback finds the figure where it looks for one. A file two questions use is carried once; where two different files are called the same, the second is named as Lambda Feedback's own exports name an image, `question_001__0001.png`. Reading an export back is unchanged, since an export already names its images this way. - A draft can freeze more than one document, which is how a sheet written as a question file and a separate solutions file is drafted: `in2lambda source add questions.docx solutions.docx` freezes them as source 1 and source 2 of the one `questions.draft.json`, and `in2lambda source add solutions.docx --draft questions.docx` adds a file to a draft already written as its next source. Every block id and line range of a source after the first carries its number - `2/b3`, `2/s10:14`, with `1/b3` meaning the `b3` it always did - and a field quoted from one records which source it came from, so that the same line number in two documents is two different places. `in2lambda source show` prints each source under its number and its name. `in2lambda spec run` runs the spec over every source: the first is laid out as its `layout` says, and in any source after it the `question` selector picks out the marker written above each question's solutions while everything else the spec picks out is a solution, paired onto the questions and parts of the first the way `in2lambda convert -a` pairs an answers file. A draft now holds `sources`, a list of `{source, hash, blocks}` in the order they were frozen, rather than those three at the top level, so a draft written before this is refused as one nothing here wrote; `in2lambda source add --start-over` freezes the document again. In the Python API, `in2lambda.source.add` takes a list of files and the draft to freeze them into; `in2lambda.source.frozen` and `in2lambda.draft.apply` hand back and take the markdown of every source rather than of one; `in2lambda.spec.fields` takes one `(blocks, markdown)` pair per source in place of its `elements` and `markdown` arguments; and `in2lambda.spec.Field` carries the number of the source its ranges are lines of, which anything constructing one has to say. - Every command that works on a draft - `in2lambda source show`, each of the `in2lambda draft` commands, `in2lambda spec run`, `in2lambda validate`, `in2lambda build` and `in2lambda render` - takes `--draft`, naming either the draft or the source it was frozen from. Left off, it uses the one draft in the current directory, and where there is more than one it is refused naming them rather than acting on whichever sorts first. `in2lambda spec run` names its SPEC from the draft's directory. The Python functions behind them take the draft's path rather than a directory: `in2lambda.source.frozen`, `in2lambda.source.show`, `in2lambda.draft.execute`, `in2lambda.draft.replay`, `in2lambda.draft.spec_command`, `in2lambda.draft.report.validate`, `in2lambda.draft.export.build` and `in2lambda.draft.export.render`. `in2lambda.source.draft_of` says where a document's draft goes and `in2lambda.source.find` is what the command line resolves `--draft` with. +- `in2lambda source add` now records the blocks nested inside a block as well as the top-level ones, so that a sheet written as one list - each question an item, its parts a list inside that item - has a block per part for a spec to select. A list item and a fenced div, which is what pandoc makes of a `\begin{solution}` environment, are the two blocks that hold blocks of their own; one holding a single element other than a list stays one block. A nested block's id is the id of the block holding it and a number, `b3.1` and `b3.2.1`, and it carries a `depth`: 1 for a top-level element, 2 for a block inside one. A block spans the blocks nested inside it, and a fenced div's lines are the ones its content stands on rather than the `:::` lines pandoc wrote around it. `in2lambda source show` prints the ids against the line each block starts on, indented two spaces for each level below the top, so a line starting a block and the first block inside it carries both ids. A spec selects a nested block by `depth`, as in `part: ListItem depth=2`, and a block whose children hold a role holds none itself, so one selector may match a block and its children both. A block with children is no longer reported as being in no field; its children are reported instead. A draft frozen from a document with nothing nested in it is written exactly as before and replays; a draft frozen before this from a document with a list in it does not replay, since freezing it now finds blocks the draft has not got, and `in2lambda source add --start-over` freezes it again. - Importing `in2lambda.katex_convert` no longer writes a file called `log` into the working directory. What it has to say about a converted expression goes to the `in2lambda.katex_convert` logger, which is silent unless the application configures logging. - The Python API is unchanged: `in2lambda.main.runner` and everything under `in2lambda.api` take the same arguments and return the same objects. diff --git a/docs/source/drafts.md b/docs/source/drafts.md index d1b0fe6..2bb0034 100644 --- a/docs/source/drafts.md +++ b/docs/source/drafts.md @@ -105,8 +105,10 @@ $ cat sheet.draft.json } ``` -A block is one top-level element pandoc found - a heading, a paragraph, a list item - with the -lines it spans and an id to quote it by. `fields` holds the questions, parts and solutions +A block is one element pandoc found - a heading, a paragraph, a list item - with the lines it +spans and an id to quote it by. A list item, or a `\begin{solution}` environment, holds elements +of its own, and each of those is a block too: `b3` holds `b3.1` and `b3.2`, and `b3` spans them. +[Specs](spec.md) says more about the nested ones. `fields` holds the questions, parts and solutions written from the sheet, and `log` holds the commands that wrote them. Both are empty until a spec or a command fills them in. @@ -143,8 +145,9 @@ b8 15 The flow rate is $Q = \pi d^2 v / 4$. ``` `in2lambda source show` prints the frozen markdown numbered, with the id of each block against -the line it starts on. A command names a block by its id, `b3`, a line of the source, `s5`, or a -range of lines, `s15:16`. +the line it starts on. A block and the first block nested inside it start on the same line, so a +line may carry several ids, and the margin is indented two spaces for each level of nesting. A +command names a block by its id, `b3`, a line of the source, `s5`, or a range of lines, `s15:16`. ## Fill the draft in from a spec diff --git a/docs/source/spec.md b/docs/source/spec.md index 478f3f6..974bc33 100644 --- a/docs/source/spec.md +++ b/docs/source/spec.md @@ -71,13 +71,14 @@ A selector is a block type, then any number of constraints: ``` The type is a pandoc element - `Header`, `Para`, `ListItem` - and may be left out to match any -block. A constraint is about one of three things: +block. A constraint is about one of four things: | Attribute | What it is | |-----------|------------| | `level` | A heading's level: `level=2` is `##`. | | `text` | The whole block as text, with the markup taken off. | | `label` | The first word of that text, which is usually what numbers a question. | +| `depth` | How deep the block sits: `depth=1` is a top-level element of the document, `depth=2` a block nested inside one. See **Nested blocks** below. | `=` asks for exactly that; `~` for a regular expression anywhere in it. `after SELECTOR,` says the block has to come after the first block that selector matches, which is how the solutions at @@ -93,6 +94,42 @@ pandoc reads the item - the marker off the first line and as much of the same wi line under it - so `strip` is only for what pandoc does not read as a marker, the `Q1. ` and the `Solution: `. +## Nested blocks + +Many sheets are written as one list: each question is an item, and the parts of a question are a +list nested inside that item. `in2lambda source add` records the blocks inside a block as well as +the top-level ones, so that a selector reaches a part. + +A list item and a fenced div - what pandoc makes of a `\begin{solution}` environment - are the +two blocks that hold blocks of their own. One holding a single element other than a list is that +element and stays one block. A nested block's id is the id of the block holding it and a number: +`b3` holds `b3.1` and `b3.2`, and `b3.2` holds `b3.2.1`. `in2lambda source show` prints the ids +against the line each block starts on, indented two spaces for each level below the top. + +A block spans the blocks nested inside it, so `depth` is what tells a question from its parts: + +```yaml +question: Para depth=2 +part: Para depth=3 +solution: Div +layout: PartSolPartSol +``` + +That spec reads a sheet whose questions are top-level items. The question is the item's own +paragraph, `b3.1`, rather than the whole item `b3`, and `after` still pairs a solution with the +part above it. + +A block whose children hold a role holds none itself. Writing `b3` into `q1.text` and `b3.2` +into `q1.p1.text` would be two fields quoted from the same lines, which no command writes: a +spec quotes the item's own paragraphs into the question and the nested items into the parts. So +one selector may match a block and its children both, and the block steps aside for them. + +A fenced div's lines are the ones its content stands on. The `:::` lines pandoc wrote around it +are pandoc's, as a list marker is, and no field quotes them. + +A block with children is not reported as being in no field. Its children are reported instead, +because a parent spans the blank lines and fences between them, which nothing can quote. + ## Predicates Some documents cannot be told apart by their text. If the questions are the paragraphs written diff --git a/in2lambda/draft/__init__.py b/in2lambda/draft/__init__.py index 2318288..bfd92f9 100644 --- a/in2lambda/draft/__init__.py +++ b/in2lambda/draft/__init__.py @@ -18,14 +18,15 @@ import in2lambda.spec from in2lambda.draft.report import _order, _where, checks, overlapping, uncovered from in2lambda.source import ( + Block, SourceError, _digest, _elements, _numbered, _require_conversion_tools, blocks, - dedented, frozen, + quoted, save, serialise, ) @@ -515,24 +516,36 @@ def _quoted( ) -> str: """Lines of one frozen source as a field takes them. - Lines quoted out of a list item are dedented by the item's own indentation, which - is the markdown's rather than the author's; the range is still the source lines. - The block the lines fall in says whether they are, rather than the text itself, so - that a paragraph reading like a list item is quoted as it is written. + Lines quoted out of a list item, or out of a block nested inside one, are dedented + by the indentation the markdown gave them rather than the author; the range is still + the source lines. The block the lines fall in says whether they are, rather than the + text itself, so that a paragraph reading like a list item is quoted as it is written. """ text = "\n".join(markdown.splitlines()[start - 1 : end]) - # Blocks do not overlap, so the one the first line falls in is the one the lines are - # part of - a nested item among them included, since only a top-level item is a - # block of its own and a range is how one of those is quoted. + # The innermost block the first line falls in, which is the last one to hold it since + # a block is written after the block it sits inside. A parent's indentation is the + # child's too, so either would dedent by the same width; the innermost is what says + # how deep the lines stand when the parent is a fenced div rather than an item. block = next( ( held - for held in draft["sources"][source - 1]["blocks"] + for held in reversed(draft["sources"][source - 1]["blocks"]) if held["start"] <= start <= held["end"] ), None, ) - return dedented(text) if block and block["type"] == "list item" else text + if block is None: + return text + return quoted( + text, + Block( + block["id"], + block["type"], + block["start"], + block["end"], + block.get("depth", 1), + ), + ) def _next(draft: dict[str, Any], prefix: str) -> str: diff --git a/in2lambda/draft/report.py b/in2lambda/draft/report.py index 2412797..f2f45ec 100644 --- a/in2lambda/draft/report.py +++ b/in2lambda/draft/report.py @@ -112,6 +112,11 @@ def uncovered(draft: dict[str, Any]) -> list[Finding]: Blocks are accounted for by the lines the fields were taken from rather than by name, so that a block `split block` has cut in two is covered by an ignore of the whole. + A block with blocks nested inside it is not reported; its children are. Such a block + spans its children and the blank lines and fences pandoc wrote between them, which no + field can quote, so reporting it would name lines nobody can account for and say + again what each child already says. + Args: draft: A draft, as `in2lambda.source.frozen` reads one. @@ -131,7 +136,16 @@ def uncovered(draft: dict[str, Any]) -> list[Finding]: } found = [] for number, source in enumerate(draft["sources"], start=1): + # A nested block's id is its parent's and a number, so a block whose id is the + # stem of another's is one holding blocks of its own. + parents = { + block["id"].rsplit(".", 1)[0] + for block in source["blocks"] + if "." in block["id"] + } for block in source["blocks"]: + if block["id"] in parents: + continue free = _runs( [ line diff --git a/in2lambda/source/__init__.py b/in2lambda/source/__init__.py index faa494e..7aa1bbf 100644 --- a/in2lambda/source/__init__.py +++ b/in2lambda/source/__init__.py @@ -4,7 +4,12 @@ take the wording out of the source rather than retype it, and a line range is only an address if the text it points into cannot move underneath it. So the document is frozen once: converted to markdown, hashed, and written down beside a ``FILE.draft.json`` -listing every top-level block with the lines it spans. +listing every block with the lines it spans. + +A block is a top-level element of the markdown, or an element nested inside one. A list +item holding a list of its own is one block per question and one per part, and the +nested ids say where each sits: ``b3`` holds ``b3.1`` and ``b3.2``, and ``b3.2`` holds +``b3.2.1``. A parent's line range spans its children's. The draft is named after the source it was frozen from, so a folder holding a term's worth of sheets holds a draft for each rather than one they take turns overwriting. @@ -25,7 +30,8 @@ import re import shutil import subprocess -from dataclasses import dataclass +import textwrap +from dataclasses import dataclass, replace from pathlib import Path from typing import Any, Optional @@ -574,7 +580,7 @@ def frozen(draft: str | Path) -> tuple[dict[str, Any], list[str]]: @dataclass class Block: - """One top-level block of a frozen source, and the lines it spans. + """One block of a frozen source, and the lines it spans. Lines are 1-based and inclusive, so `start` and `end` are the numbers :func:`show` prints beside that block's first and last line. @@ -584,10 +590,24 @@ class Block: type: str start: int end: int + depth: int = 1 + """How deep the block sits: 1 for a top-level element, 2 for a child of one.""" def to_dict(self) -> dict[str, str | int]: - """The block as it is written into the draft.""" - return {"id": self.id, "type": self.type, "start": self.start, "end": self.end} + """The block as it is written into the draft. + + A top-level block writes no ``depth``, so a document with nothing nested in it + freezes to the draft it has always frozen to and replays as it always did. + """ + written: dict[str, str | int] = { + "id": self.id, + "type": self.type, + "start": self.start, + "end": self.end, + } + if self.depth > 1: + written["depth"] = self.depth + return written def _numbered(source: int, name: str) -> str: @@ -601,7 +621,7 @@ def _numbered(source: int, name: str) -> str: def blocks(markdown: str, source: int = 1) -> list[Block]: - r"""Every top-level block of some markdown, in the order it is written. + r"""Every block of some markdown, in the order it is written. Args: markdown: A document in the dialect :func:`add` freezes to. @@ -609,15 +629,18 @@ def blocks(markdown: str, source: int = 1) -> list[Block]: but the first: a draft's second source has ``2/b1``, ``2/b2``. Returns: - One :class:`Block` per block, numbered ``b1`` onwards. The blocks do not - overlap and every line of the document falls in at most one: a block that is + One :class:`Block` per top-level element, numbered ``b1`` onwards, each followed + by the blocks nested inside it, numbered ``b1.1`` onwards. Top-level blocks do + not overlap and every line of the document falls in at most one: a block that is none of the types the agent quotes is still listed, as ``other``, rather than - leaving its lines unaddressable. + leaving its lines unaddressable. A block with children spans them. Examples: >>> from in2lambda.source import blocks >>> blocks("# Title\n\nSome words.\n") - [Block(id='b1', type='heading', start=1, end=1), Block(id='b2', type='paragraph', start=3, end=3)] + [Block(id='b1', type='heading', start=1, end=1, depth=1), Block(id='b2', type='paragraph', start=3, end=3, depth=1)] + >>> [(block.id, block.type) for block in blocks("1. Q1\n\n 1. (a)\n")] + [('b1', 'list item'), ('b1.1', 'paragraph'), ('b1.2', 'list item')] >>> [block.id for block in blocks("# Solutions\n", 2)] ['2/b1'] """ @@ -641,7 +664,9 @@ def dedented(text: str) -> str: keeps its own relative indent. Text whose first line has no marker on it comes back unchanged, but a paragraph reading like one - ``A. Smith says`` - would be dedented, so what this is called on is decided by the block's type rather than - by its text. + by its text. The same holds of a block nested inside an item: :func:`quoted` + calls this on one because its first line may carry the item's marker, and a + nested block whose first line reads like a marker is dedented too. Examples: >>> from in2lambda.source import dedented @@ -662,6 +687,35 @@ def dedented(text: str) -> str: ) +def quoted(text: str, block: Block) -> str: + r"""Some lines of a block, as a field quotes them. + + Args: + text: The lines as the source writes them. + block: The block they are the lines of, which says whether the indentation on + them is the markdown's. + + Returns: + The lines as they are, for a top-level block that is not a list item. For a list + item, or for a block nested inside one, the marker comes off the first line and + the indent the lines stand at comes off every one of them: both are what the + markdown needed to hold the item together, and four leading spaces after a blank + line are a code block wherever the field is rendered. + + Examples: + >>> from in2lambda.source import Block, quoted + >>> quoted("1. A person walks\n to the edge.", Block("b1", "list item", 3, 4)) + 'A person walks\nto the edge.' + >>> quoted(" It might have\n two lines.", Block("b1.2", "paragraph", 7, 8, 2)) + 'It might have\ntwo lines.' + >>> quoted("A. Smith says\nso.", Block("b1", "paragraph", 1, 2)) + 'A. Smith says\nso.' + """ + if block.type == "list item" or block.depth > 1: + return textwrap.dedent(dedented(text)) + return text + + def _elements(markdown: str, source: int = 1) -> list[tuple[Block, Any]]: """Every block of some markdown, each beside the panflute element it was taken from. @@ -674,22 +728,59 @@ def _elements(markdown: str, source: int = 1) -> list[tuple[Block, Any]]: document = pf.convert_text( markdown, input_format=f"{_MARKDOWN}+sourcepos", standalone=True ) - found = [span for element in document.content for span in _spans(element, pf)] + found = _walk(document.content, "b", 1, len(markdown.splitlines()), pf) + return [ + (replace(block, id=_numbered(source, block.id)), element) + for block, element in found + ] + + +def _walk(elements, prefix, depth, limit, pf): # type: ignore[no-untyped-def] + """The blocks a run of sibling elements accounts for, each parent before its children. + + Args: + elements: The elements standing side by side - the document's own, or the ones + inside one block of it. + prefix: What their ids start with: ``b`` at the top, ``b3.`` inside ``b3``. + depth: How deep they sit, counting the top-level elements as 1. + limit: The last line the final sibling may reach, which is the end of the + document at the top and the end of the parent block inside one. + pf: The panflute module, imported by the caller that has it. + """ + found = [span for element in elements for span in _spans(element, pf)] # Where no blank line separates one block from the next - a list straight after a # paragraph, a definition list - pandoc reports the first as running on into the # second's first line, so no block is allowed to reach where the next one starts, - # nor past the end of the document. - limits = [start - 1 for _, start, _, _ in found[1:]] + [len(markdown.splitlines())] - return [ - (Block(_numbered(source, f"b{number}"), kind, start, min(end, limit)), element) - for number, ((kind, start, end, element), limit) in enumerate( - zip(found, limits), 1 + # nor past the end of what holds it. + limits = [start - 1 for _, start, _, _ in found[1:]] + [limit] + walked = [] + for number, ((kind, start, end, element), stop) in enumerate(zip(found, limits), 1): + block = Block(f"{prefix}{number}", kind, start, min(end, stop), depth) + walked.append((block, element)) + walked.extend( + _walk(_children(element, pf), f"{block.id}.", depth + 1, block.end, pf) ) - ] + return walked + + +def _children(element, pf): # type: ignore[no-untyped-def] + """The elements inside a block that are blocks of their own, and none where it has any. + + A list item and a fenced Div - what pandoc makes of a ``solution`` environment - are + the two things a document nests blocks inside, and a spec reaches those blocks by + their depth. One holding a single element other than a list is that element, so it + stays one block: a sheet written without nesting freezes to the blocks it always did. + """ + if not isinstance(element, (pf.ListItem, pf.Div)): + return [] + inside = [_unwrapped(child, pf) for child in element.content] + if len(inside) == 1 and not isinstance(inside[0], (pf.BulletList, pf.OrderedList)): + return [] + return list(element.content) def _spans(element, pf): # type: ignore[no-untyped-def] - """The ``(type, start, end, element)`` quadruples one top-level element accounts for. + """The ``(type, start, end, element)`` quadruples one element accounts for. A list is several: the ticket asks for a list item, not a list, and an item spans everything nested under it. The element given back is the one that block is, past @@ -706,6 +797,20 @@ def _spans(element, pf): # type: ignore[no-untyped-def] for item in inner.content if len(item.content) ] + if isinstance(inner, pf.Div): + # A fenced Div's own position covers the `:::` lines pandoc wrote around it. + # Those are pandoc's, as a list marker is, and a field cannot quote them, so the + # lines of the Div are the ones its content stands on. + if not len(inner.content): + return [] + return [ + ( + _kind(inner, pf), + _range(inner.content[0])[0], + _range(inner.content[-1])[1], + inner, + ) + ] return [(_kind(inner, pf), *_range(element), inner)] @@ -738,6 +843,8 @@ def _kind(inner, pf) -> str: # type: ignore[no-untyped-def] if isinstance(contents[0], pf.Image): return "image" return "paragraph" + if isinstance(inner, pf.Div): + return "div" return "other" @@ -897,10 +1004,12 @@ def show(draft: str | Path) -> str: draft: The path of the draft to print. Returns: - One line per line of each frozen markdown: the id of the block starting there, - where one does, then the line number and the line itself. A draft of more than - one source heads each with its number and its name, since the line numbers - start again at 1 in every one of them. + One line per line of each frozen markdown: the ids of the blocks starting there, + where any do, then the line number and the line itself. A nested block and the + block holding it start on the same line, so a line may carry several ids, and + the margin is indented two spaces for each level of nesting below the top. A + draft of more than one source heads each with its number and its name, since the + line numbers start again at 1 in every one of them. Raises: DraftMissing: there is no draft at that path. @@ -918,12 +1027,22 @@ def show(draft: str | Path) -> str: for number, (source, markdown) in enumerate( zip(found["sources"], sources), start=1 ): - ids = {block["start"]: block["id"] for block in source["blocks"]} + starting: dict[int, list[dict[str, Any]]] = {} + for block in source["blocks"]: + starting.setdefault(block["start"], []).append(block) + # The blocks are in document order, so the ids of one line read outermost first, + # and the margin is indented by the shallowest of them: a nested block whose + # parent starts further up stands out to the right of it. + ids = { + line_number: " " * (min(block.get("depth", 1) for block in found) - 1) + + " ".join(block["id"] for block in found) + for line_number, found in starting.items() + } lines = markdown.splitlines() - margin = max((len(block_id) for block_id in ids.values()), default=0) + margin = max((len(shown) for shown in ids.values()), default=0) numbers = len(str(len(lines))) body = "\n".join( - f"{ids.get(line_number, ''):>{margin}} {line_number:>{numbers}} {line}".rstrip() + f"{ids.get(line_number, ''):<{margin}} {line_number:>{numbers}} {line}".rstrip() for line_number, line in enumerate(lines, start=1) ) printed.append( diff --git a/in2lambda/spec/__init__.py b/in2lambda/spec/__init__.py index 7bd1769..bbf07f4 100644 --- a/in2lambda/spec/__init__.py +++ b/in2lambda/spec/__init__.py @@ -47,13 +47,19 @@ from typing import Any, NamedTuple, Optional from in2lambda.filters import builtin_filters -from in2lambda.source import Block, SourceError, dedented +from in2lambda.source import Block, SourceError, quoted _KEYS = ("question", "part", "solution", "strip", "ignore", "layout", "predicates") """Everything a spec may say. Anything else in one is a typo, and is refused as one.""" -_ATTRIBUTES = ("level", "text", "label") -"""What a constraint can be about: a heading's level, a block's text, its first word.""" +_ATTRIBUTES = ("level", "text", "label", "depth") +"""What a constraint can be about. + +A heading's level, a block's text, its first word, and how deep the block sits: 1 for a +top-level element of the document, 2 for a block nested inside one. A sheet whose +questions are list items with their parts nested under them is written ``question: +ListItem depth=1`` and ``part: ListItem depth=2``. +""" _ROLES = ("ignore", "question", "part", "solution") """The roles a spec holds selectors for, in the order a block is tried against them. @@ -112,7 +118,7 @@ class Selector: def matches( self, - elements: list[Any], + elements: list[tuple[Block, Any]], index: int, pf: Any, functions: Optional[dict[str, Callable[[Any], Any]]] = None, @@ -120,7 +126,9 @@ def matches( """Whether the block at `index` is one of these. Args: - elements: Every block of the document, as the panflute element it is. + elements: Every block of the document, each beside the panflute element it + is. A constraint about ``depth`` is about the block; the rest, and a + predicate, are about the element. index: Which of them to decide about. pf: The panflute module, imported by the caller that has it. functions: The predicates the spec's file holds, as :func:`predicates` bound @@ -130,7 +138,7 @@ def matches( True if the element is of this type, meets every constraint, satisfies every predicate it calls, and comes after something the ``after`` selector matches. """ - element = elements[index] + block, element = elements[index] if self.after is not None and not any( self.after.matches(elements, earlier, pf, functions) for earlier in range(index) @@ -139,7 +147,7 @@ def matches( if self.type is not None and type(element).__name__ != self.type: return False if not all( - constraint.holds(_attribute(constraint.attribute, element, pf)) + constraint.holds(_attribute(constraint.attribute, block, element, pf)) for constraint in self.constraints ): return False @@ -188,8 +196,10 @@ class Doubled(NamedTuple): """Which block the field holds, and the lines that block was taken from.""" -def _attribute(name: str, element: Any, pf: Any) -> Optional[str]: +def _attribute(name: str, block: Block, element: Any, pf: Any) -> Optional[str]: """What a block says for one attribute, or None where it has not got one.""" + if name == "depth": + return str(block.depth) if name == "level": return str(element.level) if isinstance(element, pf.Header) else None text = pf.stringify(element).strip() @@ -634,21 +644,31 @@ def _roles( matches within the source it is run over - ``after Header text=Solutions`` is about where a block sits in its own document - so each source is decided about on its own, whatever the sources before it hold. + + A block whose children hold a role holds none itself. A parent spans its children, + so a question quoted from the whole of a list item and a part quoted from an item + nested inside it would be two fields written over the same lines, which + `in2lambda.draft.record` refuses. The spec quotes the item's own paragraph into the + question and the nested items into the parts, and one selector may match both. """ - found = [element for _, element in elements] - return [ + found = [ next( ( role for role in _ROLES if any( - selector.matches(found, index, pf, functions) + selector.matches(elements, index, pf, functions) for selector in getattr(spec, role) ) ), None, ) - for index in range(len(found)) + for index in range(len(elements)) + ] + held = {block.id for (block, _), role in zip(elements, found) if role is not None} + return [ + None if any(other.startswith(f"{block.id}.") for other in held) else role + for (block, _), role in zip(elements, found) ] @@ -718,11 +738,9 @@ def _stripped(spec: Spec, lines: list[str], block: Block) -> str: The markdown rather than the text pandoc stringifies it to, so that the maths, the emphasis and the images in a question survive into the field. """ - text = "\n".join(lines[block.start - 1 : block.end]) # The list marker and the indent under it are the markdown's, not the author's, so # they come off before the spec's patterns, which are for what is left. - if block.type == "list item": - text = dedented(text) + text = quoted("\n".join(lines[block.start - 1 : block.end]), block) for pattern in spec.strip: text = pattern.sub("", text) return text.strip() diff --git a/tests/conftest.py b/tests/conftest.py index a882ab0..118b0ea 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -52,9 +52,15 @@ def frozen_sources(folder: Path) -> list[str]: A folder holding a ``solutions.md`` beside its ``source.md`` is a sheet written as two documents - the questions, and the worked solutions separately - and freezes as - two sources, so covering that is a second file in a folder rather than a test. + two sources, so covering that is a second file in a folder rather than a test. A + folder whose spec runs over a filter's ``example.tex`` holds a ``source.tex``, which + `tests.test_spec._frozen` copies in. """ - return [name for name in ("source.md", "solutions.md") if (folder / name).is_file()] + return [ + name + for name in ("source.md", "source.tex", "solutions.md") + if (folder / name).is_file() + ] @pytest.fixture diff --git a/tests/fixtures/drafts/README.md b/tests/fixtures/drafts/README.md index 7aa85c1..8960b16 100644 --- a/tests/fixtures/drafts/README.md +++ b/tests/fixtures/drafts/README.md @@ -37,8 +37,10 @@ the fields point at. commands one at a time, and the only one whose question has two parts: both are answered by the spec's own solutions, so the `question solution` after it answers nothing, and the export carries it as a part of its own rather than dropping the wording. `nested_list` is a numbered question with two lettered parts nested inside it, quoted by line -range because only the top-level item is a block: each field is dedented by its own depth, four -spaces for the question and eight for the parts, while its range still names the source lines. +range: each field is dedented by the depth of the block its first line falls in, four spaces for +the question and eight for the parts, while its range still names the source lines. Nothing is +left in no field, because a block holding blocks of its own is not reported and each of its +children is quoted. `question_without_parts` is a question and nothing else, which the checks warn has no solution: it is here because a question with no parts is what the export has to write out as an empty part rather than as the template's. diff --git a/tests/fixtures/drafts/nested_list/report.json b/tests/fixtures/drafts/nested_list/report.json index c582b7d..ad9f6e8 100644 --- a/tests/fixtures/drafts/nested_list/report.json +++ b/tests/fixtures/drafts/nested_list/report.json @@ -1,20 +1,4 @@ [ - { - "check": "uncovered", - "field": "b2", - "level": "error", - "message": "b2 (lines 5-5, 8-8) is in no field and not marked ignore.", - "ranges": [ - [ - 5, - 5 - ], - [ - 8, - 8 - ] - ] - }, { "check": "no-solution", "field": "q1.p1", diff --git a/tests/fixtures/sources/README.md b/tests/fixtures/sources/README.md index 7e7397f..47a2889 100644 --- a/tests/fixtures/sources/README.md +++ b/tests/fixtures/sources/README.md @@ -16,6 +16,15 @@ Windows line endings, which is what a document off a teacher's machine usually h freeze to the same blocks and to a hash that `sha256sum source.md` reproduces. The `.gitattributes` at the top of the repository is what stops a checkout rewriting those endings away. +`nested_list` and `fenced_div` are the two documents that nest blocks inside blocks. +`nested_list` is a question written as a list item holding a paragraph, a list of two parts and a +heading, and pins the dotted ids, the `depth` each carries and that `b2` spans `b2.1` to `b2.4` +while `b2.2` spans `b2.2.1` and `b2.2.2`. Its last item holds one paragraph and stays one block, +which is what keeps a document with no nesting in it freezing to the blocks it always did. +`fenced_div` is a `.tex` whose `solution` environment pandoc writes as `::: {.solution}`, once +inside a list item and once holding a list of its own, and pins that a div's range is the lines +its content stands on rather than the `:::` lines around it. + To cover another construct, add a folder: one `source.md`, `source.tex` or `source.docx`, and the `expected.json` the test compares the draft's `blocks` against. diff --git a/tests/fixtures/sources/display_maths/expected.json b/tests/fixtures/sources/display_maths/expected.json index bfdb1c2..0aa844f 100644 --- a/tests/fixtures/sources/display_maths/expected.json +++ b/tests/fixtures/sources/display_maths/expected.json @@ -41,10 +41,52 @@ "start": 17, "type": "list item" }, + { + "depth": 2, + "end": 17, + "id": "b7.1", + "start": 17, + "type": "paragraph" + }, + { + "depth": 2, + "end": 21, + "id": "b7.2", + "start": 19, + "type": "display maths" + }, + { + "depth": 2, + "end": 23, + "id": "b7.3", + "start": 23, + "type": "paragraph" + }, { "end": 31, "id": "b8", "start": 25, "type": "list item" + }, + { + "depth": 2, + "end": 25, + "id": "b8.1", + "start": 25, + "type": "paragraph" + }, + { + "depth": 2, + "end": 29, + "id": "b8.2", + "start": 27, + "type": "display maths" + }, + { + "depth": 2, + "end": 31, + "id": "b8.3", + "start": 31, + "type": "paragraph" } ] diff --git a/tests/fixtures/sources/fenced_div/expected.json b/tests/fixtures/sources/fenced_div/expected.json new file mode 100644 index 0000000..8e6b087 --- /dev/null +++ b/tests/fixtures/sources/fenced_div/expected.json @@ -0,0 +1,48 @@ +[ + { + "end": 1, + "id": "b1", + "start": 1, + "type": "heading" + }, + { + "end": 7, + "id": "b2", + "start": 3, + "type": "list item" + }, + { + "depth": 2, + "end": 3, + "id": "b2.1", + "start": 3, + "type": "paragraph" + }, + { + "depth": 2, + "end": 6, + "id": "b2.2", + "start": 6, + "type": "div" + }, + { + "end": 12, + "id": "b3", + "start": 10, + "type": "div" + }, + { + "depth": 2, + "end": 10, + "id": "b3.1", + "start": 10, + "type": "list item" + }, + { + "depth": 2, + "end": 12, + "id": "b3.2", + "start": 12, + "type": "list item" + } +] diff --git a/tests/fixtures/sources/fenced_div/source.tex b/tests/fixtures/sources/fenced_div/source.tex new file mode 100644 index 0000000..e5c35b9 --- /dev/null +++ b/tests/fixtures/sources/fenced_div/source.tex @@ -0,0 +1,16 @@ +\section{Fenced divs} + +\begin{enumerate} + \item Find the load the large piston carries. + + \begin{solution} + The load is $F = pA$. + \end{solution} +\end{enumerate} + +\begin{solution} +\begin{enumerate} + \item The small piston has area $a$. + \item The large piston has area $A$. +\end{enumerate} +\end{solution} diff --git a/tests/fixtures/sources/nested_list/expected.json b/tests/fixtures/sources/nested_list/expected.json new file mode 100644 index 0000000..04f343c --- /dev/null +++ b/tests/fixtures/sources/nested_list/expected.json @@ -0,0 +1,62 @@ +[ + { + "end": 1, + "id": "b1", + "start": 1, + "type": "heading" + }, + { + "end": 12, + "id": "b2", + "start": 3, + "type": "list item" + }, + { + "depth": 2, + "end": 4, + "id": "b2.1", + "start": 3, + "type": "paragraph" + }, + { + "depth": 2, + "end": 8, + "id": "b2.2", + "start": 6, + "type": "list item" + }, + { + "depth": 3, + "end": 6, + "id": "b2.2.1", + "start": 6, + "type": "paragraph" + }, + { + "depth": 3, + "end": 8, + "id": "b2.2.2", + "start": 8, + "type": "paragraph" + }, + { + "depth": 2, + "end": 10, + "id": "b2.3", + "start": 10, + "type": "list item" + }, + { + "depth": 2, + "end": 12, + "id": "b2.4", + "start": 12, + "type": "heading" + }, + { + "end": 14, + "id": "b3", + "start": 14, + "type": "list item" + } +] diff --git a/tests/fixtures/sources/nested_list/source.md b/tests/fixtures/sources/nested_list/source.md new file mode 100644 index 0000000..b686b26 --- /dev/null +++ b/tests/fixtures/sources/nested_list/source.md @@ -0,0 +1,14 @@ +# Problem sheet 9 + +1. A person walks from the centre + to the edge of a horizontal turntable. + + 1. Find the angular speed afterwards. + + Give the units. + + 2. Find the energy lost. + + ## Ten marks each + +2. Find the pressure at the bottom of a tank. diff --git a/tests/fixtures/specs/README.md b/tests/fixtures/specs/README.md index 59cbcf3..689bb3d 100644 --- a/tests/fixtures/specs/README.md +++ b/tests/fixtures/specs/README.md @@ -2,7 +2,9 @@ Each folder here is one run of `in2lambda spec run`: a `source.md` to freeze, the `spec.yaml` to run over it, the `expected.json` the spec should leave in the draft's `fields`, and the -`uncovered.txt` of the blocks the command should report as being in no field. The test freezes +`uncovered.txt` of the blocks the command should report as being in no field. A folder holding +no document of its own is a spec for the `example.tex` the filter its `layout` names ships, which +the test copies in as `source.tex`. The test freezes the source, runs the spec, compares both, and then replays the draft from its log and checks the file is unchanged byte for byte - so a folder covers what a spec makes of a document and that it can be rebuilt from what was recorded. A folder whose layout sends two blocks to the same field @@ -24,6 +26,14 @@ calls functions from it, because what tells its questions from the paragraph abo bold each of them starts with, which is markup rather than text. Its log entry names that file and hashes it as it does the spec, which is what makes a changed predicate refuse to replay. +`parts_sep_sol_nested`, `part_part_sol_sol_nested` and `part_sol_part_sol_nested` are the three +layouts whose example nests the parts of a question inside the question's own list item. Each +runs over the example its filter ships, so the one sheet covers the filter route and the spec +route, and each pins that a `depth` constraint reaches a nested block and that a block whose +children hold a role holds none itself. `part_part_sol_sol_nested` is the one whose solutions are +a list inside a `solution` environment, so its `solution` selector names both a nested item and +the div a solution written without parts is. + `ignore_as_a_list` writes its `ignore` as two selectors under the key rather than one beside it - the title of the sheet, and the paragraph about marks - and pins that a block either of them matches is ignored. `solutions_only` is a document of nothing but solutions, which every layout diff --git a/tests/fixtures/specs/part_part_sol_sol_nested/expected.json b/tests/fixtures/specs/part_part_sol_sol_nested/expected.json new file mode 100644 index 0000000..bb08bdf --- /dev/null +++ b/tests/fixtures/specs/part_part_sol_sol_nested/expected.json @@ -0,0 +1,98 @@ +{ + "b1.ignore": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 1, + 1 + ] + ], + "value": true + }, + "q1.p1.solution": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 10, + 10 + ] + ], + "value": "$1+1=2$" + }, + "q1.p1.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 5, + 5 + ] + ], + "value": "$1+1$" + }, + "q1.p2.solution": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 12, + 12 + ] + ], + "value": "$2+2=4$" + }, + "q1.p2.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 7, + 7 + ] + ], + "value": "$2+2$" + }, + "q1.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 3, + 3 + ] + ], + "value": "Here is some interesting information. Calculate the following:" + }, + "q2.solution": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 18, + 18 + ] + ], + "value": "This is the solution." + }, + "q2.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 15, + 15 + ] + ], + "value": "This questions has no parts. What is the solution?" + } +} diff --git a/tests/fixtures/specs/part_part_sol_sol_nested/spec.yaml b/tests/fixtures/specs/part_part_sol_sol_nested/spec.yaml new file mode 100644 index 0000000..7de99cb --- /dev/null +++ b/tests/fixtures/specs/part_part_sol_sol_nested/spec.yaml @@ -0,0 +1,7 @@ +question: Para depth=2 +part: ListItem depth=2 +solution: + - ListItem depth=3 + - Div +ignore: Header +layout: PartPartSolSol diff --git a/tests/fixtures/specs/part_part_sol_sol_nested/uncovered.txt b/tests/fixtures/specs/part_part_sol_sol_nested/uncovered.txt new file mode 100644 index 0000000..e69de29 diff --git a/tests/fixtures/specs/part_sol_part_sol_nested/expected.json b/tests/fixtures/specs/part_sol_part_sol_nested/expected.json new file mode 100644 index 0000000..5df1f15 --- /dev/null +++ b/tests/fixtures/specs/part_sol_part_sol_nested/expected.json @@ -0,0 +1,98 @@ +{ + "b1.ignore": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 1, + 1 + ] + ], + "value": true + }, + "q1.p1.solution": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 8, + 8 + ] + ], + "value": "$1+1 = 2$" + }, + "q1.p1.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 5, + 5 + ] + ], + "value": "$1+1$" + }, + "q1.p2.solution": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 14, + 14 + ] + ], + "value": "$2+2=4$" + }, + "q1.p2.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 11, + 11 + ] + ], + "value": "$2+2$" + }, + "q1.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 3, + 3 + ] + ], + "value": "Here is some interesting information. Calculate the following:" + }, + "q2.solution": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 20, + 20 + ] + ], + "value": "This is the solution." + }, + "q2.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 17, + 17 + ] + ], + "value": "This questions has no parts. What is the solution?" + } +} diff --git a/tests/fixtures/specs/part_sol_part_sol_nested/spec.yaml b/tests/fixtures/specs/part_sol_part_sol_nested/spec.yaml new file mode 100644 index 0000000..c7669ef --- /dev/null +++ b/tests/fixtures/specs/part_sol_part_sol_nested/spec.yaml @@ -0,0 +1,5 @@ +question: Para depth=2 +part: Para depth=3 +solution: Div +ignore: Header +layout: PartSolPartSol diff --git a/tests/fixtures/specs/part_sol_part_sol_nested/uncovered.txt b/tests/fixtures/specs/part_sol_part_sol_nested/uncovered.txt new file mode 100644 index 0000000..e69de29 diff --git a/tests/fixtures/specs/parts_sep_sol_nested/expected.json b/tests/fixtures/specs/parts_sep_sol_nested/expected.json new file mode 100644 index 0000000..49b2b07 --- /dev/null +++ b/tests/fixtures/specs/parts_sep_sol_nested/expected.json @@ -0,0 +1,86 @@ +{ + "b1.ignore": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 1, + 1 + ] + ], + "value": true + }, + "b3.2.ignore": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 13, + 13 + ] + ], + "value": true + }, + "q1.p1.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 5, + 7 + ] + ], + "value": "Here is the first subquestion. This would map onto (a),(b),(c) etc in Lambda Feedback.\n\nIt might have multiple paragraphs." + }, + "q1.p2.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 9, + 9 + ] + ], + "value": "Here is a second subquestion." + }, + "q1.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 3, + 3 + ] + ], + "value": "This is a sample question" + }, + "q2.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 11, + 11 + ] + ], + "value": "This is a second question." + }, + "q3.text": { + "by": "tests", + "edited": false, + "layer": 1, + "ranges": [ + [ + 15, + 15 + ] + ], + "value": "This is a third question. A separate solution file (if provided) should follow the same format." + } +} diff --git a/tests/fixtures/specs/parts_sep_sol_nested/spec.yaml b/tests/fixtures/specs/parts_sep_sol_nested/spec.yaml new file mode 100644 index 0000000..156179f --- /dev/null +++ b/tests/fixtures/specs/parts_sep_sol_nested/spec.yaml @@ -0,0 +1,6 @@ +question: + - Para depth=2 + - ListItem depth=1 +part: ListItem depth=2 +ignore: Header +layout: PartsSepSol diff --git a/tests/fixtures/specs/parts_sep_sol_nested/uncovered.txt b/tests/fixtures/specs/parts_sep_sol_nested/uncovered.txt new file mode 100644 index 0000000..e69de29 diff --git a/tests/test_source.py b/tests/test_source.py index 20c9197..ab12e32 100644 --- a/tests/test_source.py +++ b/tests/test_source.py @@ -135,7 +135,37 @@ def test_source_show_numbers_the_lines_and_names_the_blocks( ] for block in blocks: assert lines[block["start"] - 1].split()[0] == block["id"] - assert sum(bool(re.match(r" *b\d+ ", line)) for line in lines) == len(blocks) + assert sum(bool(re.match(r" *b[\d.]+ ", line)) for line in lines) == len(blocks) + + +def test_source_show_indents_the_ids_of_nested_blocks( + tmp_path: Path, monkeypatch +) -> None: + """A spec names a part by the depth it sits at, so the printing has to show it.""" + shutil.copytree(SOURCES_DIR / "nested_list", tmp_path, dirs_exist_ok=True) + monkeypatch.chdir(tmp_path) + runner = CliRunner() + assert runner.invoke(cli, ["source", "add", "source.md"]).exit_code == 0 + + result = runner.invoke(cli, ["source", "show"]) + + assert result.exit_code == 0, result.output + lines = result.output.splitlines() + blocks = json.loads((tmp_path / "source.draft.json").read_text())["sources"][0][ + "blocks" + ] + for block in blocks: + assert block["id"] in lines[block["start"] - 1].split() + # A block and the first block inside it start on the same line, so both ids are + # printed against it, and the margin is indented two spaces per level below the top. + assert [lines[number - 1] for number in (1, 3, 6, 8, 10, 14)] == [ + "b1 1 # Problem sheet 9", + "b2 b2.1 3 1. A person walks from the centre", + " b2.2 b2.2.1 6 1. Find the angular speed afterwards.", + " b2.2.2 8 Give the units.", + " b2.3 10 2. Find the energy lost.", + "b3 14 2. Find the pressure at the bottom of a tank.", + ] def test_a_second_source_is_frozen_beside_the_first( diff --git a/tests/test_spec.py b/tests/test_spec.py index add10a3..191d68e 100644 --- a/tests/test_spec.py +++ b/tests/test_spec.py @@ -15,13 +15,18 @@ from typing import Any import pytest +import yaml from click.testing import CliRunner from conftest import SPECS, SPECS_DIR, frozen_sources +import in2lambda import in2lambda.draft from in2lambda.main import cli from in2lambda.source import SourceError +FILTERS = Path(in2lambda.__file__).parent / "filters" +"""Where the example each filter ships lives, for a spec folder holding no document.""" + WORKED_EXAMPLE = SPECS_DIR / "parts_sep_sol" """The case the tests below happen to use; what they check holds for any of them.""" @@ -33,8 +38,17 @@ def _frozen(folder: Path, tmp_path: Path) -> CliRunner: - """A folder's documents and its spec, copied into `tmp_path` with the sources frozen.""" + """A folder's documents and its spec, copied into `tmp_path` with the sources frozen. + + A folder holding no document of its own is a spec written for the example the filter + it names ships, which is copied in as the source: the three layouts that nest their + parts inside their questions are spec-able exactly where the filters are, so the one + sheet covers both routes. + """ shutil.copytree(folder, tmp_path, dirs_exist_ok=True) + if not frozen_sources(tmp_path): + layout = yaml.safe_load((folder / "spec.yaml").read_text())["layout"] + shutil.copy(FILTERS / layout / "example.tex", tmp_path / "source.tex") runner = CliRunner() assert ( runner.invoke(cli, ["source", "add", *frozen_sources(tmp_path)]).exit_code == 0 From cfe2a2900920f3ee1b7e3b98256d948350958736 Mon Sep 17 00:00:00 2001 From: "Peter B. Johnson" <peterbjohnson@gmail.com> Date: Mon, 21 Sep 2026 00:59:36 +0100 Subject: [PATCH 2/3] implement: Let a spec reach the items nested inside a block (t51) --- tests/fixtures/against_convert/PartsOneSol/spec.yaml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/fixtures/against_convert/PartsOneSol/spec.yaml b/tests/fixtures/against_convert/PartsOneSol/spec.yaml index 6b5394b..8925e8c 100644 --- a/tests/fixtures/against_convert/PartsOneSol/spec.yaml +++ b/tests/fixtures/against_convert/PartsOneSol/spec.yaml @@ -1,6 +1,5 @@ -question: Para +question: Para depth=1 part: ListItem solution: Div -strip: ['(?m)^::: \{\.solution\}\n', '(?m)^:::$'] ignore: Header layout: PartsOneSol From 883fc6f85933d031f7380287eae3aaa619731985 Mon Sep 17 00:00:00 2001 From: "Peter B. Johnson" <peterbjohnson@gmail.com> Date: Mon, 21 Sep 2026 01:42:55 +0100 Subject: [PATCH 3/3] implement: Let a spec reach the items nested inside a block (t51) --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ca33ad5..c379426 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,7 @@ - A draft can freeze more than one document, which is how a sheet written as a question file and a separate solutions file is drafted. `in2lambda source add questions.docx solutions.docx` freezes them as source 1 and source 2 of the one `questions.draft.json`, and `in2lambda source add solutions.docx --draft questions.docx` adds a file to an existing draft as its next source. Every block id and line range of a source after the first carries that source's number — `2/b3`, `2/s10:14` — and `1/b3` names the block `b3` names. A field quoted from a source records which source it came from, so that the same line number in two documents is two places. `in2lambda source show` prints each source under its number and its name. `in2lambda spec run` runs the spec over every source: the first source is laid out as the spec's `layout` says, and in any source after it the `question` selector picks out the marker written above each question's solutions while every other match is a solution, paired onto the questions and parts of the first source as `in2lambda convert -a` pairs an answers file. A draft now holds `sources`, a list of `{source, hash, blocks}` in the order they were frozen, in place of those three keys at the top level, so a draft written before this release is refused as a draft in2lambda did not write; `in2lambda source add --start-over` freezes the document again. Four changes to the Python API break existing scripts: `in2lambda.source.add` takes a list of files and the draft to freeze them into; `in2lambda.source.frozen` returns the markdown of every source and `in2lambda.draft.apply` takes the markdown of every source, in place of one; `in2lambda.spec.fields` takes one `(blocks, markdown)` pair per source in place of its `elements` and `markdown` arguments; and `in2lambda.spec.Field` carries the number of the source its ranges are lines of, which every caller constructing a `Field` must pass. - Every command that works on a draft takes `--draft`, naming either the draft or the source it was frozen from: `in2lambda source show`, each `in2lambda draft` command, `in2lambda spec run`, `in2lambda validate`, `in2lambda build` and `in2lambda render`. Left off, each command uses the one draft in the current directory, and where the directory holds more than one draft, the command is refused, naming them. `in2lambda spec run` resolves its SPEC from the draft's directory. The Python functions behind those commands take the draft's path in place of a directory: `in2lambda.source.frozen`, `in2lambda.source.show`, `in2lambda.draft.execute`, `in2lambda.draft.replay`, `in2lambda.draft.spec_command`, `in2lambda.draft.report.validate`, `in2lambda.draft.export.build` and `in2lambda.draft.export.render`. `in2lambda.source.draft_of` returns the path of a document's draft, and `in2lambda.source.find` resolves `--draft` for the command line. - `in2lambda source add` records the blocks nested inside a block as well as the top-level blocks, so that a sheet written as one list — each question an item, each part an item of a list inside that item — holds a block per part for a spec to select. A list item and a fenced div, which is what pandoc writes a `\begin{solution}` environment as, are the two blocks that hold blocks of their own. A list item or div holding a single element other than a list is one block. A nested block's id is the id of the block holding it and a number, such as `b3.1` and `b3.2.1`, and the block carries a `depth`: 1 for a top-level element, 2 for a block inside one. A block spans the blocks nested inside it. A fenced div spans the lines its content is written on, and not the `:::` lines pandoc wrote around it. `in2lambda source show` prints the ids against the line each block starts on, indented two spaces for each level below the top, so that a line starting a block and the first block inside it carries both ids. A spec selects a nested block by `depth`, as in `part: ListItem depth=2`. A block whose children hold a role holds no role itself, so one selector may match a block and its children. `in2lambda validate` reports the children of a block with children, in place of the block itself, as being in no field. A draft frozen from a document holding nothing nested is written as it was before this release and replays. A draft frozen before this release from a document holding a list does not replay, because freezing that document now records blocks the draft does not hold; `in2lambda source add --start-over` freezes the document again. -- A spec written before this release can match blocks it did not match before, because a selector that names no `depth` now matches a block nested inside a list item or a `solution` environment as well as a top-level block. The `PartsOneSol` example ships a `solution` environment holding three elements, two of them paragraphs, so a spec reading `question: Para` now matches those two paragraphs; the div holding them then holds no role, and the run writes two questions the document does not write and no worked solution. Write `depth=1` on each selector — `question: Para depth=1` — to keep the meaning the selector had before this release. One change to the Python API breaks existing scripts: `in2lambda.spec.Selector.matches` takes a list of `(block, element)` pairs in place of a list of panflute elements, so a caller matching a selector itself passes each `in2lambda.source.Block` beside its element. +- A spec written before this release can match blocks it did not match before, because a selector that names no `depth` now matches a block nested inside a list item or a `solution` environment as well as a top-level block. The `PartsOneSol` example ships a `solution` environment holding two paragraphs and a display maths, which pandoc writes as a `Para` as well, so a spec reading `question: Para` now matches all three of them; the div holding them then holds no role, and the run writes three questions the document does not write in place of the first question's worked solution. The second question's worked solution survives, because that `solution` environment holds one element and stays one block. Write `depth=1` on each selector — `question: Para depth=1` — to keep the meaning the selector had before this release. One change to the Python API breaks existing scripts: `in2lambda.spec.Selector.matches` takes a list of `(block, element)` pairs in place of a list of panflute elements, so a caller matching a selector itself passes each `in2lambda.source.Block` beside its element. - `in2lambda convert FILE PartsOneSol` now exports the worked solution a document writes in a `solution` environment. Pandoc writes that environment as a Div whose classes hold `solution`, and the filter recognised only a Div whose first block reads `Solution`, so a document using the environment exported every question with an empty worked solution. A Div whose first block reads `Solution` is still recognised. - Importing `in2lambda.katex_convert` no longer writes a file called `log` into the working directory. That module reports what it changed in an expression to the `in2lambda.katex_convert` logger, which is silent unless the application configures logging. - `in2lambda convert` now reads a .docx that holds an image. in2lambda looks in the document for the directories a `\graphicspath` names, and read the document as UTF-8 text to find them. A .docx is a zip file, so converting a Word document holding a figure raised `UnicodeDecodeError`. in2lambda now reads a document that is not UTF-8 text as naming no directory, which is what a .docx names.