Skip to content

feat(convert): read a table Markdown cannot express as raw storage - #206

Merged
willkg merged 8 commits into
mainfrom
raw-tables-round-trip
Sep 25, 2026
Merged

willkg merged 8 commits into
mainfrom
raw-tables-round-trip

Conversation

@willkg

@willkg willkg commented Sep 25, 2026

Copy link
Copy Markdown
Member

Closes #55.

What changes

read and export used to write every table as a GFM pipe table and drop whatever did not fit: a layout, column widths, merged cells, a headerless shape, block content in a cell. The next publish then stripped those from the page. Now a table reads back as a pipe table only when Markdown expresses all of it, and otherwise as raw storage with each cell's body as Markdown between blank lines, the convention layout cells already use:

<table data-layout="center">
<tbody>
<tr>
<td rowspan="2">

**auth** is [up](https://status.example.com)

</td>
...

The decision is an allowlist (internal/convert/storage_to_md_table.go), so markup nobody has seen yet keeps a table raw rather than being dropped. Publishing already passed a raw table through intact, so nothing changes on the write side. The plan is _plans/055_raw-tables-round-trip.md, with every amendment made during implementation at its end.

markfluence's own tables stay Markdown

Tested with a real browser save: a table create published, then edited in the Confluence editor, reads back as exactly the Markdown it was created from. The editor writes ids, a bare local-id on every paragraph, data-table-width, and on an align-start table a <colgroup> of widths it measured; read ignores all of these. The cost: a column someone resizes by hand writes the same <colgroup>, so the resize is lost on the next publish. A <colgroup> on any other layout, and any layout other than align-start, keeps a table raw, since those are visible choices markfluence never makes.

Tables markfluence did not write mostly come back raw. In a sample of 152 tables edited since June, most use the default or full-width layout. That is the trade: keeping what is on the page over pretty Markdown.

Measured, not assumed

Everything about what takes effect was probed against the live instance with ADF, and recorded in docs/confluence/storage-format.md: which table markup does anything (a <caption> leaks its text above the table; a nested table becomes something the editor cannot edit), what the editor writes on a table, and how a paragraph's alignment combines with its cell's. Only center and right do anything; left and justify do nothing, and start and end are stripped, so a paragraph saying any of those four takes its cell's alignment. A cell's shared alignment is written on the cell (<td style="text-align: center;">) so its paragraphs stay Markdown.

Property test

internal/convert/table_property_test.go generates tables from a seed and checks that each reads, gives Markdown that is a fixed point after one publish, and publishes back to the same model of what takes effect in Confluence. 3000 seeds run in every make test (about 1.5 s); FuzzTableProperty runs more on demand, and about 610,000 passed; TestTablePropertyCorpus runs the same checks over real tables from a local file. Over the 148 tables in a personal space, the only two failures were outside the table rules, and are #203 and #204. The model has its own readings of every rule it checks, so a mistake in a converter helper fails the test rather than being shared by it.

Changes outside tables

These came out of the reviews and the property test, and touch more than tables:

  • The bare local-id the editor writes is now dropped everywhere, beside ac:local-id.
  • renderRawBlock writes an element holding loose text whole, where it used to drop the text (layouts and unknown macros too).
  • In a pipe cell, renderCellLines no longer puts a <br> beside a list (it published as a blank line), and a list containing a | now keeps a table raw (it split the row and broke the whole table).
  • docs/markdown-file.md wrongly said a multi-line cell publishes back to paragraphs; it publishes line breaks, which look the same.

Found and filed, not fixed here

Not in scope

A warning when a table falls back to raw; a compact one-line form for a plain-text raw cell; an empty-header convention for headerless tables; a check lint for a <colgroup> with no layout, a <caption> or a nested table. Each waits for someone to need it.

read and export turned every table into a GFM pipe table and dropped
whatever did not fit -- a layout, column widths, merged cells, a
headerless shape -- so read, edit, update stripped a hand-built table
from its page (#55).

A table now reads back as a pipe table only when Markdown expresses all
of it, decided over an allowlist; otherwise it is written as raw storage
with each cell's body as Markdown between blank lines, the convention
layout cells already use. The attributes the editor writes on every
table it saves (ac:local-id, data-table-width, the default display
mode, align-start) are ignored in the decision. A column whose cells
disagree on alignment is now raw rather than voted on.

The raw-table escape hatch is documented, with the table markup that
takes effect in Confluence, verified against the live instance.
A markfluence table saved in the browser editor read back raw: the
editor puts a bare local-id on every paragraph and a <colgroup> of
measured pixel widths on an align-start table on any save. Both are now
ignored, the <colgroup> only on align-start, so a markfluence table
stays a pipe table after an edit in Confluence. A hand resize writes the
same <colgroup> and is lost on the next publish.

From the review, each reproduced first: loose inline content in a raw
cell is one paragraph rather than a block per element, an empty
paragraph in a raw cell stays <p />, loose text in a table wrapper is
kept, an unknown text-align value keeps a table raw, a paragraph's own
alignment beats its cell's, an empty cell does not vote on its column's
alignment, the tags allowed loose in a pipe cell are those read can
render, and renderRawBlock no longer renders a raw cell twice.
Each finding was reproduced before fixing. A cell's shared paragraph
alignment now moves onto the cell as a text-align style, so its
paragraphs stay Markdown: kept as storage, an aligned paragraph kept an
image as <ac:image>, which is never uploaded when an exported tree is
published to new pages. A nested table in a raw cell stays raw, a
paragraph that renders to nothing stays storage, two text-align
declarations or a list in an aligned column keep a table raw, the bare
local-id is dropped everywhere, and an element holding loose text is
written whole rather than a line per child.
table_property_test.go generates tables from a seed and checks that
each reads without error, gives Markdown that is a fixed point after
one publish, and publishes back to the same model of what takes effect
in Confluence. The model ignores exactly the differences read may make,
so that list is written down once. 3000 seeds run in every make test,
a fuzz target runs more, and a corpus test runs the checks over real
tables named by MF_TABLE_CORPUS.

What it found, now fixed: a "|" in a list in a pipe cell broke the
table, so such a table reads raw; a <br> beside a list in a pipe cell
published as a blank line; a trailing empty paragraph, or one of only
<br />s, was lost in a pipe cell, so it reads raw; loose text beside an
aligned paragraph was read as aligned; and an empty paragraph counted as
disagreeing with its cell's alignment. docs/markdown-file.md wrongly
said a multi-line cell publishes back to paragraphs; it publishes line
breaks, which look the same.
The table property test's model called five of the converter's own rule
helpers, so a mistake in one would have been the oracle's mistake too
and passed. The model now has its own readings of alignment, a pixel
colgroup, an empty cell and the server ids, written from what Confluence
does, and the generator now writes percentage widths and paragraph
styles that do something else, so a mutation in any of those helpers
fails the test.

Hoisting a raw cell's shared alignment moves only center and right,
spelling end as right: the values verified on a cell. Left, start and
justify do nothing and are dropped; start does not survive Confluence's
sanitizer, so hoisting it would have changed the Markdown on the next
read.
Probed on a scratch page: only center and right do anything, on a
paragraph or a cell, and a paragraph saying one overrides its cell's.
Left and justify do nothing, and start and end are stripped by the
sanitizer, so a paragraph saying any of the four is aligned by its cell
like one saying nothing. read had treated a paragraph's left as
overriding its cell (CSS, not Confluence) and a storage end as right
(ADF's name for right, not a value that means it).

normalizeCellAlign replaces the hoist: it drops every declaration that
does nothing on every path, not only when a cell's alignment can move
onto the cell, since start and end written back would be stripped and
read differently. The property test's model follows the measured rule
and has its own column-width regex.
@willkg
willkg merged commit 37d8b7a into main Sep 25, 2026
1 check passed
@willkg
willkg deleted the raw-tables-round-trip branch September 25, 2026 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Raw storage tables: document the escape hatch and make it round-trip

1 participant