Choosing Between csvkit and Miller for Everyday CSV Work

Both tools turn the command line into a decent CSV workbench, and the question of which to learn first comes up often. They are not close substitutes in how they behave. csvkit is a set of small Python programs with a friendly manner and a lot of conversion ability. Miller is a single compiled program with a record-oriented language and an eye on speed. I ran the same ten operations through both on a 200,000-row file and watched what they did to the data as well as how long they took. Some of the results were bigger than I expected, which is why the comparison is worth reading before you pick. The other tools in this space are surveyed in CSV tools worth knowing.

Ten operations, timed

Five pairs of equivalent commands, each tool's output discarded so that printing does not count:

Two groups. For the first pair, the commands that just pass rows through, csvkit is about four times slower than Miller, which is what you expect from Python against a compiled tool and does not matter at this size. For the rest, csvkit is 120 to 400 times slower, nearly a minute for a 12 MB file. Something specific is going on.

Why a minute

csvkit's commands that need to know a column's type, to sort numerically, to compute a statistic or to write JSON with numbers as numbers, read the entire file first to infer the type of every column, and only then do the work. That scan is slow in Python, because it tries several parsers on each value, including dates and times. csvcut and csvgrep need no types, so they skip it.

The inference can be switched off with -I (--no-inference), and the timing changes completely:

csvjson -I                  2.66 s
csvsort -I -c amount -r     2.28 s
csvstat -I --count          0.52 s

From 58 seconds to 2.3 seconds for the sort. The price is in the output. With no inference, every column is text, and the sort is a text sort. The top rows of csvsort -I -c amount -r began with 99.99, and the largest amount in the file is 651.04:

$ csvsort -I -c amount -r orders-200k.csv | head -3 | cut -c1-60
order_id,customer_id,zip,amount,created_at,status,country,no
7422,216899,81330,99.99,2025-06-23 17:36:38,failed,JP,rush o
16697,46867,54251,99.99,2025-04-05 11:19:19,paid,JP,

A string "99.99" sorts after "651.04" in reverse, because 9 is greater than 6. This is the failure you are warned about when you drop type inference, and it is silent. The fast flag is fine for csvstat --count or csvjson where you want the text, and wrong for sorting numbers. If you need both speed and a numeric sort, that is the point at which Miller, with its numeric sort flags, or a SQL tool, does the job without the dilemma.

Inference changes values

There is one more consequence, and it is the one I would worry about most. With inference on, csvsort does not only read the types, it writes values using them. The input timestamp was 2025-04-28 21:12:46. In the output of the typed csvsort, the same field read:

$ csvsort -c amount -r orders-200k.csv | grep -m1 '^158192,' | cut -d, -f1,5
158192,2025-04-28T21:12:46

The space became a T: csvkit parsed the string as a datetime and wrote it back in ISO 8601 form. Here that is benign, even helpful, and in general it means the output of a type-inferring command is not byte-for-byte the input. Numbers lose trailing zeros, and a date in an odd format may be rewritten or, if the sniffer is confused, read as text. csvcut, which does no inference, and Miller, which passes values through unchanged unless you modify them, both returned 2025-04-28 21:12:46 for the same row. If your pipeline compares output to input, or writes back to a system with a strict format, this matters. It does not mean csvkit is wrong; it means every conversion in the chain deserves a diff of a sample.

What each is good at

Miller's strengths are speed and range. It reads and writes CSV, TSV, JSON, JSON Lines, a name-index format, a pretty-print format and markdown, converts between them with one flag, has verbs for most record operations and an expression language for computing new fields. A single binary installs anywhere. It handles quoting correctly, and its numbers are floating point, so sums carry binary noise (3196886.2000000123) until you format them (Miller for records and pipelines).

csvkit's strengths are the odd corners. in2csv converts Excel workbooks (both .xls and .xlsx), fixed-width files given a schema, JSON and other formats to CSV, which makes it the quickest way to get an Excel sheet out (converting Excel to CSV compares it with others). csvsql runs SQL on a CSV through an in-memory SQLite and can create tables and insert into PostgreSQL or MySQL. csvstack and csvjoin combine files, csvlook renders a readable table, csvformat changes dialects, and csvclean validates (CSVLint and other validators). The documentation is excellent, and the commands are named like the Unix tools they echo, so the learning curve is low (getting started with csvkit).

A rule I would use

If the job runs in a script or touches a file over a few megabytes, use Miller. If the job is converting an Excel file, loading a table into a database or exploring a small file with a gentle interface, use csvkit. If you find yourself adding -I to csvkit to make it fast, you have chosen the wrong tool for that job, since the flag removes the thing csvkit's typed commands exist to do.

A second rule applies to both. Whatever tool you use, read identifiers as text, and diff a sample of the output against the input. The checks that apply are the same as in validating a CSV file, and the typed-versus-text trap is the subject of why a CSV file has no types and what that costs you.

When csvkit is the better pick

Pick csvkit when the file is small and the task is one-off, when you need the converters, when a colleague already knows it or when you want the SQL interface without installing a database. At 200,000 rows, a one-minute wait on a rare command is acceptable. Pick it too when its documentation and error messages matter more than speed, as when teaching, since csvkit's commands read like a tutorial.

Pick neither when the file is large and the question is analytic: DuckDB read the 2 million row version of this file and grouped it in 0.20 seconds (querying a CSV file with DuckDB instead of loading it). And do not pick on speed alone. The documentation for each is the reference for flags: csvkit and Miller.