Markdown directory tree: three pandoc readers compared

On this page, a markdown directory tree is a folder layout typed by hand as plain lines, with a pipe, a backtick and hyphens marking each branch. I typed one such tree three ways and fed the file to a 3.8.3 build of pandoc: inside a fence labeled text, as bare lines, and as a nested bullet list. Run order went markdown first, commonmark second, and commonmark_x last.

Each command went through one pass on a Mac. Quoted HTML on this page comes straight from the test logs, line breaks as pandoc set them. Nothing here checks how that HTML looks to a person; the logs hold only the text pandoc printed.

The tree file and three commands

The input is tree.md. Listed with cat -n tree.md, the file runs to line 22. Here is that text with the line numbers left off:

Fenced:

```text
docs/
|-- plans/
|   `-- doe-plan.md
`-- notes.md
```

Unfenced:

docs/
|-- plans/
|   `-- doe-plan.md
`-- notes.md

As a list:

- docs/
  - plans/
    - doe-plan.md
  - notes.md

The file types the same folder three times, under the labels Fenced:, Unfenced: and As a list:. Pandoc read it once per reader:

pandoc -f markdown -t html tree.md
pandoc -f commonmark -t html tree.md
pandoc -f commonmark_x -t html tree.md

In the log, each of these runs ends with rc 0 after an HTML fragment.

Fenced tree with a text label

From the fenced copy, markdown produced these lines:

<pre class="text"><code>docs/
|-- plans/
|   `-- doe-plan.md
`-- notes.md</code></pre>

Pandoc printed the identical four lines for commonmark and for commonmark_x. The label text ended up as a class on the <pre> tag. Inside <code>, the lines match the input, down to the three spaces between the pipe and the backtick on the third tree line.

Fence opening and closing rules live in Markdown code blocks; here the focus stays on output.

The same lines with no fence

Left bare, the four tree lines became one <p> element for all three readers. This is what commonmark produced:

<p>docs/ |-- plans/ | <code>-- doe-plan.md </code>-- notes.md</p>

In that line, the breaks between tree lines show up as single spaces. The text sitting between the backtick on the third tree line and the backtick on the fourth landed in a <code> span. All three readers did that.

Inside the span, markdown differed from commonmark and commonmark_x. In the commonmark_x span, as in the commonmark one, a space follows doe-plan.md just before </code>. Under markdown, </code> follows doe-plan.md with no space. The markdown line itself holds a non-ASCII character, so it is left out here; the next section explains why.

Dash pairs and the smart extension

Outside the code span, the two hyphen pairs set commonmark apart from markdown and commonmark_x. Under commonmark, they stayed as two hyphens each, as the line above shows. Pandoc's markdown reader swapped each pair for an en dash, code point U+2013, and commonmark_x did the same.

To see the bytes, the extra script saved the markdown output of a 4-line file, tree-unfenced.md, which holds only the bare tree lines:

pandoc -f markdown -t html tree-unfenced.md -o tree-unfenced-markdown.html
od -An -tx1 tree-unfenced-markdown.html

The markdown dump has e2 80 93 in two places; those three bytes encode U+2013 in UTF-8. The commonmark_x dump has e2 80 93 at the same two places, while the commonmark dump has 2d 2d there. Within the code span, the hyphen pair stayed 2d 2d in the three dumps.

The extension lists from pandoc match this divide: +smart appears at line 59 for markdown and at line 20 for commonmark_x, while the commonmark list carries -smart on line 20. Here smart is the name of a pandoc extension.

The extra script then flipped that extension on the bare tree:

pandoc -f commonmark+smart -t html tree-unfenced.md
pandoc -f markdown-smart -t html tree-unfenced.md
pandoc -f commonmark_x-smart -t html tree-unfenced.md

Each of these runs ended with rc 0. With commonmark+smart, the hyphen pairs outside the span became U+2013, so that line is not quoted. With markdown-smart, pandoc printed:

<p>docs/ |-- plans/ | <code>-- doe-plan.md</code>-- notes.md</p>

With commonmark_x-smart, it printed:

<p>docs/ |-- plans/ | <code>-- doe-plan.md </code>-- notes.md</p>

The markdown-smart line still has no space before </code>, so switching smart off left the spacing gap between markdown on one side and commonmark plus commonmark_x on the other in place. Other punctuation that smart changes is covered in smart quotes.

The tree as a nested list

The third copy in tree.md uses bullets indented under one another. Pandoc wrote these ten lines for it under markdown:

<ul>
<li>docs/
<ul>
<li>plans/
<ul>
<li>doe-plan.md</li>
</ul></li>
<li>notes.md</li>
</ul></li>
</ul>

Those ten lines repeat exactly in the commonmark output and in the commonmark_x output. That is three <ul> elements: the second sits inside the docs/ item and the third inside the plans/ item. The pipe and backtick marks are gone in this form, since the input never had them; the folder depth lives in the nesting instead. Indent rules for lists are explained in the guide to indenting sub-lists.

Indented four spaces with no fence

Next, the extra script ran tree-indented.md, a 4-line file where each tree line starts with four spaces:

    docs/
    |-- plans/
    |   `-- doe-plan.md
    `-- notes.md

Output from all three readers, each at rc 0, read:

<pre><code>docs/
|-- plans/
|   `-- doe-plan.md
`-- notes.md</code></pre>

The four leading spaces are gone from the output lines, and the spacing after them stayed as typed. The <pre> tag here carries no class.

Bare fence, no label

In tree-nolabel.md, 6 lines long, the opening backticks have nothing after them:

```
docs/
|-- plans/
|   `-- doe-plan.md
`-- notes.md
```

Pandoc gave rc 0 under the three readers, and printed the same four HTML lines as the indented run, starting with <pre><code>docs/. Compared with the run under Fenced tree with a text label, the <pre> tag lacks class="text" and nothing else changed.

Underscores and a star inside the fence

A deeper tree went into tree-deep.md, 8 lines long, with a file name wrapped in underscore pairs and another holding a star:

```text
doe-app/
|-- src/
|   |-- __init__.py
|   `-- core/
|       `-- doe_*_notes.md
`-- README.md
```

The three readers returned rc 0 and wrote the same HTML:

<pre class="text"><code>doe-app/
|-- src/
|   |-- __init__.py
|   `-- core/
|       `-- doe_*_notes.md
`-- README.md</code></pre>

Both __init__.py and doe_*_notes.md kept their underscores and the star, and those three outputs hold no <em> or <strong> tag. Even with markdown or commonmark_x as the reader, the hyphen pairs in this fence kept their two hyphens.

Checking the output with grep

The working directory for these commands was the test folder. Put your file name in place of tree.md to try them.

pandoc -f markdown -t html tree.md -o tree-out-markdown.html
pandoc -f commonmark -t html tree.md -o tree-out-commonmark.html
pandoc -f commonmark_x -t html tree.md -o tree-out-commonmark_x.html
grep -c '<pre class="text">' tree-out-markdown.html
grep -c '<pre class="text">' tree-out-commonmark.html
grep -c '<pre class="text">' tree-out-commonmark_x.html
grep -c '<ul>' tree-out-markdown.html
grep -c '<ul>' tree-out-commonmark.html
grep -c '<ul>' tree-out-commonmark_x.html
grep -c '<code>-- doe-plan.md </code>' tree-out-markdown.html
grep -c '<code>-- doe-plan.md </code>' tree-out-commonmark.html
grep -c '<code>-- doe-plan.md </code>' tree-out-commonmark_x.html

1. Start with the pandoc lines; each wrote one file and returned rc 0.
2. Run the <pre class="text"> greps. They printed 1 per file.
3. Run the <ul> greps. They printed 3 per file.
4. Run the greps for the code span with a space before </code>. That printed 0 for the markdown file, 1 for the commonmark file and 1 for the commonmark_x file.

Run through diff, the markdown and commonmark files differed at one line, line 7, the bare tree paragraph. Between the commonmark_x file and the commonmark file, diff flagged line 7 alone as well. For a filename line above a code block, see the sibling post on code block file names.

Where Sashmark fits

Nobody can install Sashmark yet; this desktop app is still being built. Its Round-Trip design would take a correction typed onto the rendered text and save a file that differs at that correction alone. A second idea on the drawing board, Changes View, would mark each passage an agent replaced while the file sat unread, git repo or none.

It is in development and has not been released. The app itself has no AI features. See the planned features, the help pages and the roadmap for where each part stands.

More at sashmark.com.