feat(convert): read a table Markdown cannot express as raw storage - #206
Merged
Merged
Conversation
read and export turned every table into a GFM pipe table and dropped whatever did not fit -- a layout, column widths, merged cells, a headerless shape -- so read, edit, update stripped a hand-built table from its page (#55). A table now reads back as a pipe table only when Markdown expresses all of it, decided over an allowlist; otherwise it is written as raw storage with each cell's body as Markdown between blank lines, the convention layout cells already use. The attributes the editor writes on every table it saves (ac:local-id, data-table-width, the default display mode, align-start) are ignored in the decision. A column whose cells disagree on alignment is now raw rather than voted on. The raw-table escape hatch is documented, with the table markup that takes effect in Confluence, verified against the live instance.
A markfluence table saved in the browser editor read back raw: the editor puts a bare local-id on every paragraph and a <colgroup> of measured pixel widths on an align-start table on any save. Both are now ignored, the <colgroup> only on align-start, so a markfluence table stays a pipe table after an edit in Confluence. A hand resize writes the same <colgroup> and is lost on the next publish. From the review, each reproduced first: loose inline content in a raw cell is one paragraph rather than a block per element, an empty paragraph in a raw cell stays <p />, loose text in a table wrapper is kept, an unknown text-align value keeps a table raw, a paragraph's own alignment beats its cell's, an empty cell does not vote on its column's alignment, the tags allowed loose in a pipe cell are those read can render, and renderRawBlock no longer renders a raw cell twice.
Each finding was reproduced before fixing. A cell's shared paragraph alignment now moves onto the cell as a text-align style, so its paragraphs stay Markdown: kept as storage, an aligned paragraph kept an image as <ac:image>, which is never uploaded when an exported tree is published to new pages. A nested table in a raw cell stays raw, a paragraph that renders to nothing stays storage, two text-align declarations or a list in an aligned column keep a table raw, the bare local-id is dropped everywhere, and an element holding loose text is written whole rather than a line per child.
table_property_test.go generates tables from a seed and checks that each reads without error, gives Markdown that is a fixed point after one publish, and publishes back to the same model of what takes effect in Confluence. The model ignores exactly the differences read may make, so that list is written down once. 3000 seeds run in every make test, a fuzz target runs more, and a corpus test runs the checks over real tables named by MF_TABLE_CORPUS. What it found, now fixed: a "|" in a list in a pipe cell broke the table, so such a table reads raw; a <br> beside a list in a pipe cell published as a blank line; a trailing empty paragraph, or one of only <br />s, was lost in a pipe cell, so it reads raw; loose text beside an aligned paragraph was read as aligned; and an empty paragraph counted as disagreeing with its cell's alignment. docs/markdown-file.md wrongly said a multi-line cell publishes back to paragraphs; it publishes line breaks, which look the same.
The table property test's model called five of the converter's own rule helpers, so a mistake in one would have been the oracle's mistake too and passed. The model now has its own readings of alignment, a pixel colgroup, an empty cell and the server ids, written from what Confluence does, and the generator now writes percentage widths and paragraph styles that do something else, so a mutation in any of those helpers fails the test. Hoisting a raw cell's shared alignment moves only center and right, spelling end as right: the values verified on a cell. Left, start and justify do nothing and are dropped; start does not survive Confluence's sanitizer, so hoisting it would have changed the Markdown on the next read.
Probed on a scratch page: only center and right do anything, on a paragraph or a cell, and a paragraph saying one overrides its cell's. Left and justify do nothing, and start and end are stripped by the sanitizer, so a paragraph saying any of the four is aligned by its cell like one saying nothing. read had treated a paragraph's left as overriding its cell (CSS, not Confluence) and a storage end as right (ADF's name for right, not a value that means it). normalizeCellAlign replaces the hoist: it drops every declaration that does nothing on every path, not only when a cell's alignment can move onto the cell, since start and end written back would be stripped and read differently. The property test's model follows the measured rule and has its own column-width regex.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #55.
What changes
readandexportused to write every table as a GFM pipe table and drop whatever did not fit: a layout, column widths, merged cells, a headerless shape, block content in a cell. The next publish then stripped those from the page. Now a table reads back as a pipe table only when Markdown expresses all of it, and otherwise as raw storage with each cell's body as Markdown between blank lines, the convention layout cells already use:The decision is an allowlist (
internal/convert/storage_to_md_table.go), so markup nobody has seen yet keeps a table raw rather than being dropped. Publishing already passed a raw table through intact, so nothing changes on the write side. The plan is_plans/055_raw-tables-round-trip.md, with every amendment made during implementation at its end.markfluence's own tables stay Markdown
Tested with a real browser save: a table
createpublished, then edited in the Confluence editor, reads back as exactly the Markdown it was created from. The editor writes ids, a barelocal-idon every paragraph,data-table-width, and on analign-starttable a<colgroup>of widths it measured;readignores all of these. The cost: a column someone resizes by hand writes the same<colgroup>, so the resize is lost on the next publish. A<colgroup>on any other layout, and any layout other thanalign-start, keeps a table raw, since those are visible choices markfluence never makes.Tables markfluence did not write mostly come back raw. In a sample of 152 tables edited since June, most use the
defaultorfull-widthlayout. That is the trade: keeping what is on the page over pretty Markdown.Measured, not assumed
Everything about what takes effect was probed against the live instance with ADF, and recorded in
docs/confluence/storage-format.md: which table markup does anything (a<caption>leaks its text above the table; a nested table becomes something the editor cannot edit), what the editor writes on a table, and how a paragraph's alignment combines with its cell's. Onlycenterandrightdo anything;leftandjustifydo nothing, andstartandendare stripped, so a paragraph saying any of those four takes its cell's alignment. A cell's shared alignment is written on the cell (<td style="text-align: center;">) so its paragraphs stay Markdown.Property test
internal/convert/table_property_test.gogenerates tables from a seed and checks that each reads, gives Markdown that is a fixed point after one publish, and publishes back to the same model of what takes effect in Confluence. 3000 seeds run in everymake test(about 1.5 s);FuzzTablePropertyruns more on demand, and about 610,000 passed;TestTablePropertyCorpusruns the same checks over real tables from a local file. Over the 148 tables in a personal space, the only two failures were outside the table rules, and are #203 and #204. The model has its own readings of every rule it checks, so a mistake in a converter helper fails the test rather than being shared by it.Changes outside tables
These came out of the reviews and the property test, and touch more than tables:
local-idthe editor writes is now dropped everywhere, besideac:local-id.renderRawBlockwrites an element holding loose text whole, where it used to drop the text (layouts and unknown macros too).renderCellLinesno longer puts a<br>beside a list (it published as a blank line), and a list containing a|now keeps a table raw (it split the row and broke the whole table).docs/markdown-file.mdwrongly said a multi-line cell publishes back to paragraphs; it publishes line breaks, which look the same.Found and filed, not fixed here
1.,#,>) becomes that block on publish.Not in scope
A warning when a table falls back to raw; a compact one-line form for a plain-text raw cell; an empty-header convention for headerless tables; a
checklint for a<colgroup>with no layout, a<caption>or a nested table. Each waits for someone to need it.