CSV
Exporting Large Tables to CSV Without Running Out of Memory
Export a 2,000,000 row table to CSV six ways: peak memory ranged from 11 MB to 2 GB, time from 1.6 s to 14 s. fetchall and pandas are the ones that run out.
9 min read
Topic
Data Wrangling
Take messy CSV, JSON and spreadsheets and land them in tables without losing a row or a character on the way.
Excel
Using the Sheets API From Python to Read and Write Data
Use the Google Sheets API from Python: create a service account, share the sheet, read and write ranges in batches, set value options and handle quota errors.
7 min read
CSV
Exporting a CSV With Headers From SQLite on the Command Line
Export SQLite to CSV with a header row from the sqlite3 shell: CRLF endings by default, NULL against empty string, types and 2,000,000 rows in 3.5 s.
10 min read
Excel
Importing Google Sheets Data Into a Database Reliably
Import Google Sheets data into a database reliably: read with a service account, pad rows, check the contract, load through staging and report rejected rows.
7 min read
CSV
Exporting a MongoDB Collection to CSV With mongoexport
mongoexport writes CSV only if you name the fields. See arrays, ObjectIds, nulls, the empty file and exit 0 on a typo, and 500,000 documents in 2.7 s.
10 min read
CSV
Exporting an Oracle Query to CSV With SQLcl From the Shell
Export an Oracle query to a quoted CSV with SQLcl: set the CSV format, spool to a file, fix date and number formats, and run it from a shell script.
8 min read
ExcelGuide
Google Sheets Alongside Databases Without the Mess
Run Google Sheets beside a database without chaos: import and publish data, use the API, compare QUERY with SQL, respect limits, plan the exit.
17 min read
CSV
Exporting SQL Server Results to CSV With bcp and sqlcmd
How to export a SQL Server query to a real CSV with bcp: add the header row, quote fields yourself, fix dates and NULLs, and pick the right encoding.
10 min read
CSV
Exporting MySQL Query Results to CSV With INTO OUTFILE
INTO OUTFILE writes on the server and its popular recipe corrupts newlines and NULLs: 4 of 8 rows read back. A variant reads 7, a client script reads 8.
11 min read
CSV
Exporting a PostgreSQL Query to CSV With psql and COPY
Export a PostgreSQL query to CSV with psql: NULL against empty string, delimiter and quoting options, encodings, why variables fail and the file left behind.
9 min read
CSVGuide
Exporting Query Results to CSV From Five Major Databases
The same eight awkward rows exported from PostgreSQL, MySQL, SQLite, DuckDB and MongoDB, then read back: four tools return all 8, one recipe returns 4.
25 min read
CSV
Writing Data Quality Checks for CSV Files in dbt and DuckDB
Read a CSV in a dbt model with DuckDB and test it with unique, accepted_values, relationships and SQL tests: 6 of 8 failed on a faulty file.
12 min read
Get new articles by email
We send one email when a new article is published. No spam.