CSV
Filtering CSV Rows From the Command Line With Miller
Filter, sort, compute and summarize CSV from the shell with Miller's filter, put and then-chains, and see how it treats empty cells, leading zeros and sums.
8 min read
Topic
CSV
The format everyone exports and nobody specifies. Delimiters, quoting, encodings and loading it into a database intact.
CSV
Getting Started With csvkit for Everyday CSV Tasks
csvkit gives you csvcut, csvgrep, csvlook, csvstat, csvsort and csvsql with real CSV parsing. See the everyday commands, inference traps and speed cost.
8 min read
CSV
Using awk on CSV Files and Where It Gets Dangerous
awk -F, sums and groups a CSV in one pass when nothing is quoted. See the cases where it silently returns wrong fields, and what FPAT does and does not fix.
8 min read
CSV
Selecting CSV Columns With cut and Why Quotes Break It
cut -d, -f picks CSV columns until a quoted comma shifts every field after it. See the failure, a guard that detects it, and CSV-aware replacements.
8 min read
CSV
Previewing a CSV in the Terminal With column and less
Look at a CSV in the terminal with head, column -t, less -S, csvlook, qsv table and Miller, and learn where each one misleads you on quotes and encodings.
8 min read
Editor's pick
CSVGuide
Working With CSV Files in the Shell Without Opening Excel
A practical guide to CSV in the shell: coreutils, awk, csvkit, Miller, qsv and DuckDB compared on 2,000,000 rows, with the quoting rule and a triage script.
22 min read
CSV
Python CSV Newlines and the Blank Line Bug on Windows
Why a CSV written by Python shows a blank line after every row on Windows, how newline='' prevents it, and how to check the bytes. Reproduced without Windows.
8 min read
CSV
Python CSV Error Field Larger Than Field Limit and the Fix
Fix _csv.Error field larger than field limit (131072). Tell a genuinely large cell from an unclosed quote, raise the limit safely and find the bad record.
8 min read
CSV
Reading a CSV With Inconsistent Column Counts in Python
Find the rows in a CSV whose field count differs from the header, learn why they happen, and quarantine or repair them in Python without losing data.
8 min read
CSV
Why pandas Turns ZIP Codes Into Numbers and How to Stop It
pandas read_csv infers a column of digits as integers, so 02134 becomes 2134. See the cause, the 190,451 broken codes in 2M rows, and the fixes that work.
7 min read
YAMLGuide
Converting YAML to and From JSON CSV XML and Excel
What survives when YAML becomes JSON, CSV, XML or Excel and back, with a tested loss matrix, the type traps in each direction and a conversion checklist.
19 min read
CSV
Parsing Dates While Reading a CSV With pandas Correctly
Use parse_dates and date_format in pandas read_csv, then handle mixed formats and day-first files, where format="mixed" guessed two conventions in one column.
8 min read
Get new articles by email
We send one email when a new article is published. No spam.