CSVGuide
Exporting Query Results to CSV From Five Major Databases
The same eight awkward rows exported from PostgreSQL, MySQL, SQLite, DuckDB and MongoDB, then read back: four tools return all 8, one recipe returns 4.
25 min read
Topic
CSV
The format everyone exports and nobody specifies. Delimiters, quoting, encodings and loading it into a database intact.
CSV
Writing Data Quality Checks for CSV Files in dbt and DuckDB
Read a CSV in a dbt model with DuckDB and test it with unique, accepted_values, relationships and SQL tests: 6 of 8 failed on a faulty file.
12 min read
CSV
Making a CSV Contract With Your Data Provider in Writing
What to agree with whoever sends you CSV files, as a document plus a manifest your code can check: six deliveries, six different rejections, 193,708 rows.
12 min read
CSV
Testing a CSV Export for Truncated Last Lines in Python
A cut-off CSV loaded in pandas without a warning, last row 200000,Ana,Smith,...,2. Check the final byte, the last record's width, quotes, row count and gzip.
11 min read
XML
XML Versus CSV for Exchanging Tabular Data Between Systems
A partner asks for XML for a flat table. Measured on 100,000 rows, XML was 2.7 times the CSV size, though gzip nearly erased the gap. When each is right.
6 min read
CSV
Detecting Columns That Changed Type Between CSV Deliveries
April's load failed on qty 1.5, but six columns had drifted. Compare per-column kinds of value between two CSV deliveries, not just inferred types.
11 min read
CSV
Validating a CSV File Against a Lookup Table With SQL
Find CSV values missing from a lookup table with an anti-join, keep line numbers right, avoid the NOT IN trap with NULLs and see a foreign key's error.
10 min read
CSV
Checking That Every Date in a CSV File Parses Correctly
Check every date in a CSV with a strict parse and a round trip, then ranges. strptime accepted 2025-3-4, DuckDB cast kept the date of 2025-03-14 10:30.
10 min read
CSV
Finding Duplicate Keys in a CSV File Before an Import
Find repeated ids in a CSV before a primary key rejects the load, with SQL, pandas and uniq -d, line numbers and an exact versus conflicting split.
9 min read
CSV
Validating CSV Columns With Python and pydantic Models
Validate each CSV row against a pydantic model with types, literals, patterns and a cross-field rule: 193,708 rows in 0.95 s, with file line numbers.
10 min read
CSV
Writing a Schema for a CSV With Frictionless Data in Python
Infer a Table Schema, fix it by hand, add unique, enum and pattern constraints and validate with Frictionless: 10 errors for 8 faults in 193,708 rows.
11 min read
Excel
Converting Google Sheets to CSV or Excel With a Script
Export a Google Sheet to CSV or xlsx from a script using the export URL, the Drive API or gspread, choose the sheet by gid and handle sharing and auth safely.
6 min read
Get new articles by email
We send one email when a new article is published. No spam.