Converting a CSV Into a YAML List With Correct Types

A CSV file is a table of text, and a YAML list is a list of records with types. Converting one into the other looks like a loop over rows, and the loop is easy. The part that needs thought is the types: the CSV says nothing about which column is a number, which is a boolean and which merely looks like a number. This post converts a small file twice, once without types and once with them, and explains which to choose. It is part of the cluster on converting YAML to and from JSON, CSV, XML and Excel, and the general version for configuration seeds is converting CSV to YAML for configuration and seed data.

The input and the quick conversion

id,name,zip,active,ratio,note
1,Alice,02134,yes,0.5,
2,Bob,10001,no,1,hi: there

The shortest conversion reads rows as dictionaries and dumps them:

import csv
import io
import yaml

rows = list(csv.DictReader(io.StringIO(raw)))
print(yaml.safe_dump(rows, sort_keys=False))

- id: '1'
  name: Alice
  zip: '02134'
  active: 'yes'
  ratio: '0.5'
  note: ''
- id: '2'
  name: Bob
  zip: '10001'
  active: 'no'
  ratio: '1'
  note: 'hi: there'

This is correct, and probably not what you want. csv.DictReader returns strings for every cell because that is all a CSV holds. The dumper therefore quoted '1', '0.5', 'yes' and the empty '', since a plain 1 would load back as an integer and the library preserves what it was given. A program reading this YAML gets text everywhere, and an if active: test treats 'no' as true because it is a non-empty string. The output preserves the CSV faithfully, which makes it an accurate copy of the problem.

Reloading shows the same thing from the other side: yaml.safe_load(...)[0] returned {'active': 'yes', 'id': '1', 'name': 'Alice', 'note': '', 'ratio': '0.5', 'zip': '02134'}, every value a string.

Give columns types

Decide the type of each column and write it down. A small mapping from column name to a converting function does the work:

conv = {
    "id": int,
    "ratio": float,
    "active": lambda v: v.lower() in ("yes", "true", "1"),
}
typed = [
    {k: (None if v == "" else conv.get(k, str)(v)) for k, v in r.items()}
    for r in rows
]
print(yaml.safe_dump(typed, sort_keys=False))

Columns not in the map fall back to str, and an empty cell becomes None. The output:

- id: 1
  name: Alice
  zip: '02134'
  active: true
  ratio: 0.5
  note: null
- id: 2
  name: Bob
  zip: '10001'
  active: false
  ratio: 1.0
  note: 'hi: there'

Now id is an integer, active is a boolean, ratio is a float and the empty note is null. Three details are worth noticing.

The zip code stayed text. It was never in the map, so it stayed a string, and the dumper wrote '02134' with quotes. Without them, 02134 would load as the integer 1116, because YAML 1.1 reads a leading zero as octal. The quoting is the library protecting your data, and the case is worked in a number turning into something else.

**The second ratio became 1.0.** The CSV said 1 and the map said float, so the YAML says 1.0. That is the right value for a float column and a visible difference from the source text. If the consumer wants the original digits, keep the column as text, or use decimal.Decimal and a representer.

The note was quoted. The value hi: there contains a colon followed by a space, which would start a mapping if written plainly. The library quoted it. A hand-built note: hi: there is a parse error, which is the reason to use a library at all.

Where the type map comes from

Writing conv by hand is fine for six columns. For more, derive it from something authoritative instead of from the data:

• A schema. If the data has a JSON Schema, a Pydantic model or a database table definition, convert using its types. Validating YAML configuration with Pydantic shows a model that does the conversion and the validation in one step.

• A header convention. Some exports encode types in the header (id:int, ratio:float). Parse the suffix and strip it.

• Inference, with a human check. Pandas can guess types with read_csv, and a guess is fine when you review the result; do not let it run unattended on data you do not own.

Avoid inference from the first rows in a pipeline. A column that is all digits in the sample may hold N/A in row 40,000, and a conversion that guessed int fails there at 2 a.m. or, worse, converts the column to floats and loses digits. The same problem is the subject of why a CSV file has no types and what that costs you.

Booleans deserve a decision

yes, Y, true, 1, TRUE and x all mean true in some spreadsheet somewhere. The lambda above accepts three. Make the list explicit and make the function raise on a value it does not know, instead of treating everything else as false:

def to_bool(v):
    v = v.strip().lower()
    if v in ("yes", "true", "1"):
        return True
    if v in ("no", "false", "0"):
        return False
    raise ValueError(f"not a boolean: {v!r}")

A raised error on row 812 is better than a quiet false that hides a typo.

Writing the YAML for people

The default safe_dump output is acceptable. Four options make it read better for a file humans will keep:

• sort_keys=False keeps the column order from the CSV, which is the order the author chose.

• allow_unicode=True keeps accented names readable.

• width controls folding of long strings.

• A header comment saying where the file came from, added after dumping, tells the next reader not to edit it by hand if it is generated.

For a large CSV, stream rather than build the list: write each record with its own yaml.safe_dump([record]) appended to the file, or use JSON Lines if the consumer allows it. A YAML list of a hundred thousand records takes noticeably longer to load than the CSV did.

When a CSV should not become a YAML list

The data is tabular and large. Keep it as CSV or Parquet. YAML adds a key to every value and turns a 20 MB file into a much larger one.

There is no stable key. A YAML list of records without an identifying field is hard to diff and merge. Make sure one column is unique, and use it as a mapping key (alice: then the fields) when the consumer looks records up by that key.

The CSV is dirty. Mixed date formats, stray whitespace and inconsistent booleans should be cleaned before the conversion, not encoded into YAML as they are. Working with CSV files in the shell covers the usual fixes.

Someone edits the CSV in Excel every week. Then the CSV is the source and the YAML is a build output. Generate it in a build step so it never drifts.

The csv module is documented in the Python standard library reference and the PyYAML dump options in the PyYAML documentation. To see what a given YAML list looks like as a table, try the CSV and JSON converter.