Converting YAML to a Markdown Table for Docs and READMEs

Documentation that repeats configuration goes stale. A README with a table of services, ports and owners is correct on the day it is written and wrong a month later, because the YAML file that feeds the deploy was edited and nobody touched the table. The fix is to make the table a product of the YAML. This post writes a short converter from a YAML list to a Markdown table, covers the characters that break it, and shows how to keep the result in sync. It is part of the cluster on converting YAML to and from JSON, CSV, XML and Excel; the CSV version of the same idea, aimed at spreadsheets, is converting YAML to CSV for review.

The input

Two records, with a list, a null and a value that contains a pipe:

- {name: api, port: 8080, note: "a|b", tags: [web, public]}
- {name: worker, port: 9000, note: null, tags: []}

The converter

A Markdown table is three kinds of line: a header, a separator of dashes and the rows. Each is cells joined by pipes.

import yaml

rows = yaml.safe_load(open("services.yaml"))
cols = list(rows[0])


def cell(v):
    if v is None:
        return ""
    if isinstance(v, list):
        v = ", ".join(map(str, v))
    return str(v).replace("|", "\\|")


print("| " + " | ".join(cols) + " |")
print("|" + "|".join("---" for _ in cols) + "|")
for r in rows:
    print("| " + " | ".join(cell(r.get(c)) for c in cols) + " |")

Its output:

| name | port | note | tags |
|---|---|---|---|
| api | 8080 | a\|b | web, public |
| worker | 9000 |  |  |

Rendered, that is a four-column table with two rows. The decisions in cell are the ones to read closely.

Pipes. The pipe separates columns, so a pipe inside a value ends the cell early and shifts everything after it. In the GitHub table syntax a backslash before the pipe makes it literal, which is what replace("|", "\\|") does. This is the only escaping rule the format requires. If the value contains a backtick span, the pipe still needs the backslash on GitHub, even inside the backticks.

Lists. A list in a cell would print as ['web', 'public'], the Python repr, which looks like a mistake in documentation. Joining with a comma and a space gives web, public. For long lists, use <br> as the separator, since many renderers allow that tag inside a table cell, though not all do.

Null and empty. None becomes an empty cell, and so does an empty list, which join turned into the empty string. Showing None, null or [] in documentation is noise. If a blank is ambiguous for your readers, write a dash.

Newlines. A value with a line break would end the row. Replace newlines with a space or <br> before output.

Column choices

Using list(rows[0]) takes the columns from the first record, in YAML order. That has two weaknesses. A key present only in later records is dropped. And the first record sets the order, which may not be the order readers want. Both are cheap to fix:

cols = []
for r in rows:
    for k in r:
        if k not in cols:
            cols.append(k)

That collects every key in first-seen order. To control the order and the headings, pass a list of pairs instead, such as [("name", "Service"), ("port", "Port")], and read the headings from it. Documentation tables deserve human headings; the YAML key names belong in code.

Alignment is optional and a matter of taste. Colons in the separator row align a column: |:---|---:| left-aligns the first and right-aligns the second. Numbers read better right-aligned, and the converter can emit that when every value in a column is numeric. The table source does not have to look tidy, since renderers align it; padding cells to equal width makes the raw file readable in a diff, which some teams prefer.

Keeping it in sync

A generated table should not be edited, and it should not be allowed to go stale. Three arrangements work.

1. Marker comments in the README. The converter replaces everything between <!-- table:start --> and <!-- table:end -->, leaving the rest of the document as written. Humans own the prose, the script owns the table.

2. A generated file that the README includes or links to. If your docs system supports includes, keep services.md entirely generated.

3. A CI check. Run the converter in CI, compare its output with the committed file and fail on a difference with a message that says to run the script. This is the step that makes the first two reliable, since nobody remembers.

The CI check is a few lines in any shell: regenerate to a temporary file and run diff. The same pattern keeps other derived files honest, such as JSON generated from YAML (see converting YAML to JSON in Python).

Other generators

A Markdown table is also reachable without a script. A general tool such as the Markdown table generator builds one from pasted data; yq can emit CSV (yq -o=csv), which many Markdown tools convert; and pandas has DataFrame.to_markdown(), which needs the tabulate package. Use pandas if you already load data into it, and the plain script if you want no dependency beyond PyYAML. The behavior of each with a pipe in the data differs, so test with a value that contains one.

For tables that live in documentation sites with their own components, such as MDX or reStructuredText, generate the target syntax instead. The same loop works with different delimiters.

Rendering differences

"Markdown" is a family. The table syntax comes from GitHub Flavored Markdown, and the original Markdown has no tables. Most renderers (GitHub, GitLab, many static site generators) support the pipe form. Some need an extension switched on. If your table renders as raw pipes, the renderer does not support the syntax, or there is a blank line missing before the header row. Always put a blank line before a table, because without one it is treated as a continuation of the preceding paragraph.

Cells cannot contain block elements. A list or a code fence inside a cell does not work, and a link works if written inline. Inline code works with the pipe caveat above. For long text, the table is the wrong container; use headings, or a definition list.

When a generated Markdown table is the wrong choice

The table is wide. Eight columns of text do not fit on a screen, and Markdown tables cannot wrap columns intelligently in a README viewed on a phone. Use a list of records, one heading per record.

The data is nested. A table needs flat records. Flatten nested keys with dotted names, or document each group separately.

The audience reads the YAML anyway. If the readers are the same people who edit the YAML, a table duplicates the file and adds a build step. Link to the file.

The values change per environment. A single table cannot show production and staging values together without becoming unreadable. Generate one per environment, or document the defaults only.

The table syntax is defined in the GitHub Flavored Markdown specification, and the PyYAML loader in the PyYAML documentation. The to_markdown method is documented in the pandas reference.