
Handling CSV Files Too Big to Open in Excel or Python
A decision guide for CSV files too big to open: look, count, sample, query in place, chunk, split, compress or convert, with numbers from a 1.3 GB file.
Tag
102 posts

A decision guide for CSV files too big to open: look, count, sample, query in place, chunk, split, compress or convert, with numbers from a 1.3 GB file.

A tested beginner guide to XML: where you will meet it in data work, what it does better than JSON, how it is structured, and the first checks to run.

See what an XLSX workbook really contains, from cell types and shared strings to dates, formulas, styles and limits, and what each part means for your data.

A beginner guide to getting a MySQL server running: choose a version, install, secure, connect, create users and fix the first errors.

The JSON grammar fits on a page, and parsers still disagree on trailing commas, NaN, duplicate keys, big numbers and more. A 26-case test across five parsers.

Control what goes into a shared spreadsheet: templates, data validation, tables, protection, error flags and import checks that stop bad data at the source.

Build CSV import and export into an application: staged imports with dry runs, streamed and background exports, column names, round trips and safety.

Build clear Excel charts from database data: shape the query, pick the chart, keep ranges live, avoid misleading axes, add sparklines, export images.

Eight messy CSV files solved end to end with a check each: mixed dates, unquoted commas, multi-line notes, display formats, headers, bad quotes, footnotes.

Nine analysis moves for a CSV in SQL: cross tabs, period comparisons, concentration, seasonality, distribution shape, sampling, charts and reconciliation.

How to load CSV into BigQuery, Snowflake and ClickHouse: stage the file, declare the schema, pick an error policy, load once and check the counts afterward.

Connect Excel to PostgreSQL, MySQL and SQL Server with connectors and ODBC, using read only accounts, parameters, safe credentials, refresh and speed checks.