XML
Streaming XML Into PostgreSQL Without a DOM Using COPY
Parse a 127 MB XML file with iterparse and feed two tables through COPY. 400,000 orders loaded in 6.5 seconds and 44 MB, against 38 seconds for row inserts.
6 min read
Tag
bulk-load
44 posts
CSV
Why SQLite Stores Your CSV Numbers as Text and How to Fix It
SQLite .import stores every CSV value as text, so ORDER BY puts 10.25 before 2.75 and amount > 9 matches every row. See the cause and three ways to fix it.
8 min read
CSV
Turning a Folder of CSV Files Into a SQLite Database
One table per CSV file, with safe names and types inferred from the whole file. A short Python script that keeps ZIP codes as text and refuses ragged files.
9 min read
CSV
Loading Many CSV Files Into One SQLite Database in Bash
A bash loop that imports monthly CSV files into one SQLite table, records which file each row came from, and refuses to load a file twice.
7 min read
CSV
Fixing Expected Columns but Found Errors in SQLite Imports
SQLite prints expected N columns but found M and loads the row anyway, with exit status zero. See each import message and what it did to the data.
7 min read
CSV
Querying a CSV File in SQLite Without Importing It
Compile SQLite's CSV virtual table, query a file in place, and compare the cost with importing it and with DuckDB, including the typed schema surprises.
6 min read
CSV
Importing a CSV Into an Existing SQLite Table Step by Step
Importing into a table that holds data goes wrong four ways: duplicate keys, missing columns, extra columns and a new order. See each and the staging fix.
7 min read
CSV
Setting Column Types When SQLite Imports a CSV File Safely
sqlite3 .import makes untyped columns. Create the table yourself, choose between INTEGER, TEXT and STRICT, and see what a ZIP code and a bad price do in each.
7 min read
CSV
Using the sqlite3 dot import Command With a CSV File
Every option of the sqlite3 .import command that matters for CSV, from --csv and --skip to --schema temp and piped input, with the output of each.
7 min read
Editor's pick
CSVGuide
Importing CSV Files Into SQLite From the Command Line
The sqlite3 .import command loads a CSV in seconds and reports problems as warnings with exit status zero. Import safely with types, checks and a swap.
20 min read
CSV
Fixing Data Truncated Warnings in a MySQL CSV Load
LOAD DATA LOCAL clips long strings and out of range numbers to the nearest value and only warns. See each warning, the checks that find them, and the fix.
9 min read
CSV
Importing a Large CSV Into MySQL Without Locking the Table
Measure what other sessions see while LOAD DATA runs on InnoDB, whether chunking helps, and how a build and swap with RENAME TABLE avoids any visible window.
8 min read