Markdown link in heading: pandoc output compared

Pandoc kept a link inside a heading under both readers tested here, markdown and commonmark. What split them was the id attribute: markdown wrote one from the heading words, and commonmark wrote none.

I ran each command once on a macOS computer that had pandoc 3.8.3 installed. Quoted below is the raw HTML that pandoc wrote out when given a markdown link in heading text. How a program would show that HTML, or what a click would do, is outside this post's scope.

The test file

The lead test used one input, link-heading.md. Running cat link-heading.md printed:

# Notes for Jane Doe

## Read [the plan](/blog/plan) first

## [Setup](/blog/setup)

See [the heading above](#read-the-plan-first).

That gives a level-one heading with plain text, a level-two heading with a link between other words, a level-two heading made of a link and nothing else, and a body sentence that points at #read-the-plan-first. Spec lines 1043 and 1044 (CommonMark 0.31.2) state that ATX heading text is parsed as inline content, and lines 796 and 797 name links as one kind of inline content.

Ids the markdown reader added

This was the first run:

pandoc -f markdown -t html link-heading.md

<h1 id="notes-for-jane-doe">Notes for Jane Doe</h1>
<h2 id="read-the-plan-first">Read <a href="plan.md">the plan</a>
first</h2>
<h2 id="setup"><a href="setup.md">Setup</a></h2>
<p>See <a href="#read-the-plan-first">the heading above</a>.</p>

Under markdown, the second heading kept its <a> element and came out with id="read-the-plan-first". In the markdown id, the words the plan from the link text made it in, while the target plan.md was left out. Pandoc also split that <h2> across two output lines, while the input heading was a single line.

The third heading, which holds a link and no other text, became an <h2> with id="setup" under markdown. Again markdown took that id from the link words, and the <a> element remained inside the heading. The plain first heading came out with id="notes-for-jane-doe" under markdown, so in this run the heading with no link and the two linked headings were treated alike: each one got an id.

Pandoc ties these ids to its auto_identifiers extension. Line 5 of the list pandoc prints for markdown is +auto_identifiers, so markdown has it enabled before you add any flag. With -f markdown-auto_identifiers, an extra run of this file gave three headings and no id attribute on any of them. How pandoc turns heading words into an id is left to heading anchor links.

The body sentence came out with href="#read-the-plan-first". In this markdown output, the second <h2> carries that same name as its id.

Bare headings from the commonmark run

The second run swapped the reader:

pandoc -f commonmark -t html link-heading.md

<h1>Notes for Jane Doe</h1>
<h2>Read <a href="plan.md">the plan</a> first</h2>
<h2><a href="setup.md">Setup</a></h2>
<p>See <a href="#read-the-plan-first">the heading above</a>.</p>

Under commonmark, both links remained inside their headings. Not one heading in this commonmark fragment carries an id attribute. The second <h2> also stayed on a single line in this output.

The body sentence still came out with href="#read-the-plan-first", the same as under markdown. Pandoc copied that href from the source under both readers. In this commonmark fragment, though, no element has read-the-plan-first as its id.

Turning on ids for commonmark

Thirty lines make up the commonmark list that pandoc printed. The extra test grepped it for a line reading exactly +auto_identifiers or -auto_identifiers and found neither, with grep exit status 1. For a control, that grep pointed at the markdown list matched line 5.

The lead test asked for the extension anyway:

pandoc -f commonmark+auto_identifiers -t html link-heading.md

The extension auto_identifiers is not supported for commonmark.
Use --list-extensions=commonmark to list supported extensions.

Pandoc exited with status 23, and the log holds no HTML for that run. In this test, adding auto_identifiers to commonmark ended the run with that message.

A link mid-heading and a heading with two links

The extra test brought two more inputs. Running cat linkhead-mid.md printed one heading with words before and after the link:

## Ask [Jane Doe](https://example.com/jane) before the review

With -f markdown the output was:

<h2 id="ask-jane-doe-before-the-review">Ask <a
href="https://example.com/jane">Jane Doe</a> before the review</h2>

Under markdown, words from before the link, inside it and after it all went into that id. The URL did not appear in the markdown id. In this output pandoc wrapped the line inside the opening <a tag itself, so href starts the second line. Under commonmark, the same file gave:

<h2>Ask <a href="https://example.com/jane">Jane Doe</a> before the
review</h2>

The commonmark output had no id, and pandoc broke the line at a different spot from the markdown run.

Next, cat linkhead-two.md printed a heading with two links joined by a word:

## [Plan](/blog/plan) and [Budget](/blog/budget)

The markdown run gave:

<h2 id="plan-and-budget"><a href="plan.md">Plan</a> and <a
href="budget.md">Budget</a></h2>

In the markdown output, both link texts and the word between them went into id="plan-and-budget", and neither file name did. Under commonmark, both links survived and the heading had no id:

<h2><a href="plan.md">Plan</a> and <a href="budget.md">Budget</a></h2>

A brace id on a linked heading

The third extra input put a brace id after the linked heading text. Running cat linkhead-custom.md printed:

## Read [the plan](/blog/plan) first {#plan-first}

See [the plan heading](#plan-first).

With -f markdown, pandoc wrote:

<h2 id="plan-first">Read <a href="plan.md">the plan</a> first</h2>
<p>See <a href="#plan-first">the plan heading</a>.</p>

Under markdown, pandoc dropped the braces from the heading text and used plan-first as the id where read-the-plan-first had been. The link stayed put. Line 27 of the list for markdown reads +header_attributes. Brace ids get a full treatment in custom heading IDs.

Under commonmark, the same file gave:

<h2>Read <a href="plan.md">the plan</a> first {#plan-first}</h2>
<p>See <a href="#plan-first">the plan heading</a>.</p>

In the commonmark output, {#plan-first} stayed inside the <h2> as literal characters, and that heading got no id.

Spotting a missing target with grep

These steps ran in the extra test, from its test folder. Put your file names where these ones appear.

1. Write each fragment to a file with these two commands:

   pandoc -f markdown -t html link-heading.md -o linkhead-out-md.html
   pandoc -f commonmark -t html link-heading.md -o linkhead-out-cm.html
   

2. Count lines that carry an id: on the markdown output, grep -c 'id="' linkhead-out-md.html printed 3. On the commonmark output, linkhead-out-cm.html, that count was 0, with grep exit status 1.

3. Find every line that names the target: on the markdown output, grep -n 'read-the-plan-first' linkhead-out-md.html matched line 2, the <h2> with that id, and line 5, the body link.

4. Run step 3 on the other file. On the commonmark output, linkhead-out-cm.html, it matched line 4 alone, which is the body link. With no heading line in those matches, the commonmark file has no element carrying the name the link points to.

If the same notes also hold a table with pictures in its cells, images inside table cells covers what pandoc wrote for those.

Where Sashmark fits

Sashmark aims to take a correction you type directly on the page you are reading, with nothing else to open. A Round-Trip is part of that aim: one fixed word, one changed spot in the saved file, and no other difference. Changes View is in the same plan; it would flag each passage an agent altered after your earlier pass, with git present or absent.

It is in development and has not been released. The app itself has no AI features. See the planned features, the help pages and the roadmap for where each part stands.

More at sashmark.com.