CSV
Importing Only Some CSV Columns Into a PostgreSQL Table
COPY cannot skip file columns. Compare two ways to load three of eight columns of a 2,000,000 row CSV, with timings, and see why cut breaks on quotes.
6 min read
Library
Everything you need to level up your coding skills.
CSV
PostgreSQL Invalid Byte Sequence for UTF8 in a CSV Import
COPY stops with an invalid byte sequence error when a CSV is Latin 1. Fix it with the ENCODING option or client_encoding, and avoid the SQL_ASCII trap.
7 min read
CSV
PostgreSQL COPY Error Extra Data After Last Expected Column
The COPY error means a row has more fields than the table has columns. See the four usual causes, how to find the line, and the fix for each one.
7 min read
CSV
Loading a CSV With a Header Row Into PostgreSQL Tables
Skip the header, or better, check it. See what HEADER true, HEADER match and a column list do, and the errors when the file and the table disagree.
7 min read
CSV
Psql copy Versus Server Side COPY and Which One You Need
The two commands differ in which machine reads the file. See the permission errors, the path errors and the timings, and choose the one that fits.
7 min read
CSV
Using COPY to Load a CSV File Into a PostgreSQL Table
The COPY command in practice: the options that matter, the two ways to feed it, what its row count means, and why one bad value undoes the whole load.
7 min read
Editor's pick
CSVGuide
How to Load a CSV File Into PostgreSQL Without Losing Rows
Load a CSV into PostgreSQL with COPY, see what makes it fast, and learn what each failure means, with a script that loads in one transaction.
20 min read
CSV
Checking a CSV for Invalid UTF8 Before You Load It
Find the lines and bytes that break a UTF-8 load with iconv, isutf8, grep or a short Python script, and see how PostgreSQL and DuckDB report them.
7 min read
CSV
Handling Emoji in CSV Imports Without Losing Characters
MySQL utf8 columns hold three bytes per character, so an emoji becomes a question mark during a CSV load with only a warning. See the fix and the checks.
7 min read
CSV
Why Your CSV Has Question Marks Where Names Should Be
Question marks in a CSV are lost data, not a display bug. See where the substitution happens, count the damage and decide whether the export is broken.
7 min read
CSV
Removing Null Bytes and Control Characters From a CSV
A NUL byte can cut a PostgreSQL load short while Python keeps it. Find them with grep, strip them from 130 MB in under a second, and keep tabs and newlines.
6 min read
CSV
How to Open a UTF8 CSV in Excel Without Garbled Text
Excel on Windows often shows UTF-8 accents as à symbols. Add a BOM when you write the file, or import it with the right encoding, and check the bytes first.
7 min read
Get new articles by email
We send one email when a new article is published. No spam.