Skip to content
datahacks.dev
JSON

Converting JSON to YAML on the Command Line Safely

Convert JSON to YAML with yq, Python's yq and a PyYAML one-liner, and see which strings get corrupted by YAML 1.1 readers: NO, yes, 12:30, 0123 and 1e3.

7 min read
Summarize in AI
Flat illustration of a tree of nested objects beside the document it came from, with the JSON logo.

JSON to YAML looks like a harmless change of punctuation: braces out, indentation in. Almost every tool does it in one command. The risk is not in the structure, which maps one to one, but in the strings. YAML lets scalars go unquoted, and what an unquoted scalar means depends on the version of the YAML rules the reader follows. A string that was "NO" in your JSON can come out of a conversion as NO, and a reader on the other side may decide that is the boolean false. I converted a document of awkward strings with three tools and loaded each result into two YAML parsers, to see what survives.

The commands

Three tools you are likely to meet, with the command for each:

Shell
yq -p=json -o=yaml . file.json              # mikefarah/yq v4 (Go)
yq -y . file.json                           # kislyuk/yq (Python; wraps jq)
python -c 'import sys,json,yaml; print(yaml.safe_dump(json.load(sys.stdin), sort_keys=False, allow_unicode=True))' < file.json

The name yq belongs to two unrelated programs, and a script that says yq runs whichever is installed first on the PATH. The Go one takes a path expression and an output format flag (-o=yaml, -P to pretty-print), and the Python one takes jq syntax and -y for YAML output. Scripts copied from a blog post fail with confusing errors when the wrong one is installed, so check yq --version first: the Go tool prints "yq (https://github.com/mikefarah/yq/) version v4...".

For one-off conversions in a pipeline, jq cannot write YAML. The Python one-liner needs only PyYAML, and its sort_keys=False matters, as the last section shows.

What the tools wrote

The same JSON, three renderings of the awkward strings:

Text
JSON value        mikefarah yq v4     Python yq / PyYAML
"NO"              NO                  'NO'
"yes"             yes                 'yes'
"1.10"            "1.10"              '1.10'
"0123"            "0123"              '0123'
"1e3"             "1e3"               '1e3'
"2026-10-26"      "2026-10-26"        '2026-10-26'
"12:30"           12:30               '12:30'
"line1\nline2"    |- block (2 lines)   'line1<blank>  line2' (quoted)

The Go tool follows YAML 1.2, where only true, false and null (and their capitalizations) are special, so NO, yes and 12:30 are plain strings and need no quotes. It does quote what even 1.2 would read as a number or a date ("1.10", "0123", "1e3", "2026-10-26"). The Python tools follow YAML 1.1, where yes, no, on, off and the colon-separated 12:30 have special meanings, so they quote them all.

Both outputs are valid YAML. Which is right depends on who reads it.

Loading the results back

I loaded both files with PyYAML (1.1 rules, the parser behind countless Python programs and Ansible) and ruamel.yaml (1.2 rules), and compared with the original JSON:

Text
file from mikefarah yq    PyYAML:  country NO -> False, answer yes -> True, time 12:30 -> 750
                          ruamel:  identical to the JSON
file from Python yq       PyYAML:  identical to the JSON
                          ruamel:  identical to the JSON

The unquoted NO became boolean false, yes became true, and 12:30 became the integer 750 (YAML 1.1 reads it as a base 60 number: 12 minutes 30 seconds). The old name for this is the Norway problem: a list of country codes in which NO quietly turns into false. A conversion that is correct under 1.2 corrupted data for every consumer that still uses a 1.1 parser, and these include large parts of the Python and Ruby ecosystems, Ansible and a long tail of configuration loaders. The quoting-everything tools are safe in both worlds, and the file is only slightly noisier.

The rule that follows: know the YAML version of the reader, and when you do not, quote conservatively. In Go yq you can force quoting for chosen values with style="double" assignments, and with PyYAML the safe dumper already does it.

The opposite mismatch exists. PyYAML wrote the string "1e3" bare as 1e3, because its YAML 1.1 resolver requires a dot in a float and does not see 1e3 as a number. A 1.2 reader does: ruamel loaded the bare 1e3 as 1000.0. So the file from a PyYAML dump, read by a 1.2 parser, changes a string into a number. In my tests the ruamel dumper quoted it ('1e3'), the Python yq quoted it, and PyYAML's own dump did not. If the writer and the reader use different versions, look at exactly this kind of value: strings that look like numbers in scientific notation, and strings with leading zeros (0123 is octal 83 in 1.1 and the integer 123 in 1.2, so both dumpers quoted it).

Key order

JSON objects have no defined key order, though parsers preserve it, and people read the order. PyYAML sorts the keys on output by default:

Python
yaml.safe_dump(doc)                      # keys alphabetical: debug, empty, env, list, name, ...
yaml.safe_dump(doc, sort_keys=False)     # keys in the JSON's order: name, port, debug, ...

The default looks neat and destroys the author's grouping: name ends up in the middle, and ZEBRA sorts before alpha because uppercase sorts first. For configuration files read by humans, always pass sort_keys=False. The Go yq and the Python yq preserved the JSON order in my test.

Layout details that matter

A few output choices change how the YAML reads, without changing the data:

  • Block versus quoted multi-line strings. The Go tool wrote the newline string as a block scalar (multi: |- followed by two lines), which is easy to read. PyYAML wrote it as a single-quoted string with a blank line standing for the newline ('line1 blank line line2'), which is correct and hard to read. Both loaded back to "line1\nline2".

  • Indentation of lists. Python's dumper writes list items at the same indent as the parent key (tags: then - a), and yq writes them indented two spaces. Both are valid; a linter in your CI may care.

  • Long strings wrap. PyYAML wraps plain and quoted scalars at 80 columns by default, folding on spaces. A 150-character sentence turned into two lines in the file. Pass width=1000 to keep strings on one line if a diff should show whole-sentence changes.

  • Anchors. When the same Python list object appears twice in a structure, PyYAML emits &id001 and *id001 aliases. JSON loaded with json.load creates separate objects, so it did not happen in my conversions, but data built in code can trigger it. A dumper subclass that overrides ignore_aliases to return True prevents it.

  • Special numbers. float("inf") is written .inf and 1e21 as 1.0e+21, both readable by YAML parsers and neither valid in JSON.

A converter is not a validator

I made the test file slightly wrong on the first attempt: my shell's echo expanded \n into a real line break inside a JSON string, which makes the text invalid JSON (control characters must be escaped). Both yq tools converted it without complaint. If you rely on the conversion step to tell you that the input is bad JSON, you will be disappointed. Validate first (jq empty file.json, or python -m json.tool file.json > /dev/null; fixing JSON parse errors when reading files explains the messages), then convert, and read the YAML back with the parser that will consume it, comparing to the JSON. A dozen lines of Python do it:

Python
import json, yaml
a = json.load(open("y.json"))
b = yaml.safe_load(open("y.yaml"))
print({k: (a[k], b[k]) for k in a if a[k] != b[k]})     # empty dict means identical

An empty result is what you want. Run it with each parser that the file will meet.

One more asymmetry: YAML has comments and JSON does not (JSON has no comments), so a YAML file converted to JSON and back loses every comment. Treat the YAML as the source and the JSON as a generated artifact, never the reverse.

Going the other way

YAML to JSON has its own traps: dates become date objects that json.dumps cannot serialize, yes becomes true, and anchors expand. That direction is covered in converting YAML to JSON for tools that need it. For a wider view of all the formats, see converting JSON to CSV, XML, YAML and Excel.

When converting to YAML is not worth it

The consumer reads JSON. Most tools that accept YAML accept JSON too, since JSON is nearly a subset of YAML 1.2, and Kubernetes takes JSON manifests as they are. "Nearly" matters, as the next article shows for parsers that follow YAML 1.1.

The file is generated and never edited. YAML earns its place when humans edit the file. For machine output, JSON has fewer ways to be misread.

You cannot name the reader's YAML version. If the file passes through unknown loaders, quote every string and test it, or use JSON.

Details of the quoting rules are in the YAML 1.2 specification, and the yq commands are documented at mikefarah.gitbook.io/yq. Paste a suspect JSON document into the JSON formatter to validate it before you convert.

Also inYAML
Share

Keep reading

Get new articles by email

We send one email when a new article is published. No spam.

Unsubscribe in one click. No spam, ever.