Tag

conversion

49 posts

CSV

Converting a CSV to Parquet and Why You Might Want To

One DuckDB statement turned a 130 MB CSV into a 34 MB Parquet file; typed columns, dictionary encoding and min/max statistics explain the faster queries.

10 min read

JSON

Converting YAML to JSON for Tools That Need It Safely

Convert YAML to JSON with PyYAML and ruamel.yaml, and handle dates, yes and no, octal numbers, non-string keys, anchors, multiple documents and floats.

8 min read

JSON

Converting JSON to YAML on the Command Line Safely

Convert JSON to YAML with yq, Python's yq and a PyYAML one-liner, and see which strings get corrupted by YAML 1.1 readers: NO, yes, 12:30, 0123 and 1e3.

7 min read

YAML

Round Tripping YAML Without Losing Comments or Order

Edit a YAML file in code and keep its comments, anchors and quotes. PyYAML drops them, ruamel.yaml and yq keep them, and each changes some whitespace.

6 min read

CSV

Using Mlr Miller for Records and Pipelines on the Shell

Miller (mlr) treats CSV rows as named records and chains verbs with then. Real runs for sorting, grouping, new columns, JSON conversion and splitting files.

6 min read

CSV

Fixed Width Files and Converting Them to CSV Safely

Convert a fixed width file to CSV by slicing each line at known positions, check every line length, and handle implied decimals, padding and wide characters.

7 min read

Excel

Batch Converting XLS Files to XLSX With LibreOffice

Convert a folder of old XLS files to XLSX with LibreOffice headless, Python or Excel automation, and check what formatting and formulas survive.

6 min read

YAML

Converting YAML to a Markdown Table for Docs and READMEs

Generate a Markdown table from a YAML list in Python, escaping pipes, joining lists, handling blanks and keeping the table in sync with the YAML in CI.

6 min read

YAML

Converting an Excel Sheet of Settings Into YAML Using Python

Read a key, value and type sheet with openpyxl, build nested YAML from dotted keys, and keep versions, dates and blanks from changing on the way.

7 min read

YAML

Converting YAML to XML With Python and Valid Names

Convert YAML to XML with xmltodict, using @ keys for attributes, repeated elements for lists, and checks for invalid names and the single-root rule.

7 min read

YAML

Converting a CSV Into a YAML List With Correct Types

Turn CSV rows into a YAML list of mappings in Python, keeping zip codes as text and giving integers, floats, booleans and empty cells the right types.

6 min read

YAML

Converting YAML to CSV for Review in a Spreadsheet

Flatten a YAML list of service records into a CSV a reviewer can sort and filter, handling nested keys, list values, missing fields and Excel encoding.

6 min read