
How to Detect the Encoding of a CSV File Before You Load It
Four encoding detectors gave four different answers on the same six files. See which ones to trust, how to cross check them, and when only the sender knows.
Tag
590 posts

Four encoding detectors gave four different answers on the same six files. See which ones to trust, how to cross check them, and when only the sender knows.

Why accents, euro signs and emoji break in CSV files, how to diagnose each symptom from the bytes, and one routine that converts, loads and checks a file.

Write CSV that survives every reader. Wrap fields with commas, quotes or line breaks, double inner quotes, and round trip test it against four readers.

A YAML debugging guide: the error catalog across three parsers, a worked session through five stacked mistakes, silent failures and tools to prevent them.

Query an Excel file straight from T-SQL with OPENROWSET and the ACE provider, enable ad hoc distributed queries safely, and load into a staging table.

Find a file's delimiter in four steps, from looking at the bytes to a quote aware check, and see where Python, DuckDB and qsv sniffers disagree.

Set up PostgreSQL for development: choose native or Docker, pick a version, create a user, connect, find the config files and fix common errors.

Use the SQL Server Import and Export Wizard to load an Excel sheet, fix the ACE provider and 64-bit errors, and stop it from guessing column types wrongly.

A CSV has no flag that says whether row one is a header, so readers guess. Test four files, set the header explicitly, and handle duplicate names.

Why XML fails with Premature end of data in tag: truncated downloads, cut gzip streams and empty responses, how each message differs, and what to check.

MySQL Workbench cannot import XLSX directly. Convert to CSV for the wizard or LOAD DATA, or use a Python script, and avoid the strict mode and date traps.

How to read a JSON parse error: what the position means in Python, Node, PHP and jq, twelve common causes, and a method that finds the bad byte fast.