Reading CSV Files in R With readr and fread Safely

R has three CSV readers that people actually use, and they disagree on the same file. Base R's read.csv is always there and is slow on a large file. readr::read_csv is the tidyverse reader, with a column specification you can inspect. data.table::fread is the fast one, with a guessing engine of its own. Which one you pick matters less than what you tell it about the columns, because their type guessing leads to different answers on the same data. This article runs all three on the series' test files and on 2,000,000 rows, shows where they differ, and says which settings I would always set. The comparison with other languages is in the guide.

Reading a file with quoting

All three parse quoted commas and line breaks. read_csv returns a tibble and reports its guesses:

  people = {
    d <- read_csv(path, show_col_types = FALSE)
    print(as.data.frame(d)); cat("note[3] =", encodeString(d$note[3]), "\n")
    spec_csv(path) |> print()
  },

Rscript lab.R versions x

R 4.5.3 | readr 2.2.0 | data.table 1.18.6.1 

Rscript lab.R people ../data/people.csv

  id        name   city               note
1  1   Ana Silva Lisbon          likes tea
2  2 Smith, Jane Boston       said "hello"
3  3     Bo Chen   Oslo line one\nline two
4  4   Cy Dubois   Lyon               <NA>
note[3] = line one\nline two 
cols(
  id = col_double(),
  name = col_character(),
  city = col_character(),
  note = col_character()
)

The tibble has Smith, Jane in one cell, said "hello" with plain quotes, and the third note shown as line one\nline two because I printed it with encodeString. The empty note of the last record is <NA>: readr treats an empty field as a missing value by default. spec_csv() shows what it guessed, id = col_double() and three col_character(). The spec is the thing to read after the first load of any new file, and to copy into your script when it is right.

ZIP codes: three readers, two answers

A file of ZIP codes, two with leading zeros:

  zips = {
    cat("readr:\n");      print(read_csv(path, show_col_types = FALSE))
    cat("fread:\n");      print(fread(path))
    cat("base R:\n");     print(read.csv(path))
    cat("readr, zip as character:\n"); print(read_csv(path, col_types = cols(zip = col_character()), show_col_types = FALSE))
    cat("fread, colClasses:\n"); print(fread(path, colClasses = c(zip = "character")))
  },

Rscript lab.R zips ../data/zips.csv

readr:
# A tibble: 4 × 3
     id zip   state
  <dbl> <chr> <chr>
1     1 02134 MA   
2     2 90210 CA   
3     3 00501 NY   
4     4 10001 NY   
fread:
      id   zip  state
   <int> <int> <char>
1:     1  2134     MA
2:     2 90210     CA
3:     3   501     NY
4:     4 10001     NY
base R:
  id   zip state
1  1  2134    MA
2  2 90210    CA
3  3   501    NY
4  4 10001    NY
readr, zip as character:
# A tibble: 4 × 3
     id zip   state
  <dbl> <chr> <chr>
1     1 02134 MA   
2     2 90210 CA   
3     3 00501 NY   
4     4 10001 NY   
fread, colClasses:
      id    zip  state
   <int> <char> <char>
1:     1  02134     MA
2:     2  90210     CA
3:     3  00501     NY
4:     4  10001     NY

readr guessed zip as <chr> and kept 02134 and 00501. Its guessing is conservative about leading zeros. fread made the column <int>, 2134 and 501, and so did base R's read.csv. So the same file gives text in the tidyverse and numbers in the other two, and code that works on one will quietly break on the other. It is the same trap that pandas and every other typed reader sets. The fix for all three is to declare the column: col_types = cols(zip = col_character()) in readr, and colClasses = c(zip = "character") in fread (or read.csv). The last two blocks of the output show both, with 02134, 90210, 00501, 10001 in a character column. For a table with dozens of columns, col_types = cols(.default = col_character()) followed by explicit conversions is the safest start.

Blank values are not the same everywhere

A blank cell in a text column is another place where the readers diverge. The same ZIP file with the second ZIP missing:

  blank = {
    d <- read_csv(path, col_types = cols(zip = col_character()), show_col_types = FALSE); print(d); cat("is.na(zip):", is.na(d$zip), "\n")
    f <- fread(path, colClasses = c(zip = "character")); print(f); cat("fread zip == '':", f$zip == "", " is.na:", is.na(f$zip), "\n")
  },

Rscript lab.R blank ../data/zips-blank.csv

# A tibble: 3 × 3
     id zip   state
  <dbl> <chr> <chr>
1     1 02134 MA   
2     2 <NA>  CA   
3     3 00501 NY   
is.na(zip): FALSE TRUE FALSE 
      id    zip  state
   <int> <char> <char>
1:     1  02134     MA
2:     2            CA
3:     3  00501     NY
fread zip == '': FALSE TRUE FALSE  is.na: FALSE FALSE FALSE 

With col_character(), readr turned the blank into NA (is.na(zip) printed FALSE TRUE FALSE). fread with colClasses = "character" kept it as an empty string: f$zip == "" was FALSE TRUE FALSE and is.na(f$zip) was FALSE FALSE FALSE. A later filter(!is.na(zip)) works in the first and drops nothing in the second. Both are defensible, and both have to be handled explicitly. na = c("", "NA") in readr and na.strings = "" in fread make the choice yours.

Ragged rows and what each reader says

The ragged file has a record with an extra field, a trailing comma, a short record and a record split across two lines:

  ragged = {
    cat("readr:\n"); d <- read_csv(path, show_col_types = FALSE); print(nrow(d)); print(problems(d))
    cat("fread:\n"); r <- tryCatch(fread(path), warning = function(w) conditionMessage(w), error = function(e) conditionMessage(e)); print(if (is.character(r)) strsplit(r, "\n")[[1]][1] else r)
    cat("base R:\n"); r <- tryCatch(read.csv(path), error = function(e) conditionMessage(e)); print(if (is.character(r)) r else nrow(r))
  },

Rscript lab.R ragged ../data/ragged2.csv

readr:
[1] 8
# A tibble: 4 × 5
    row   col expected  actual    file                      
  <int> <int> <chr>     <chr>     <chr>                     
1     3     5 4 columns 5 columns /b/lab/data/py/ragged2.csv
2     5     5 4 columns 5 columns /b/lab/data/py/ragged2.csv
3     6     3 4 columns 3 columns /b/lab/data/py/ragged2.csv
4     8     1 4 columns 1 columns /b/lab/data/py/ragged2.csv
fread:
[1] "Stopped early on line 3. Expected 4 fields but found 5. Consider fill=TRUE. First discarded non-empty line: <<2,Bo,Oslo,likes coffee, no sugar>>"
base R:
[1] 8
Warning message:
One or more parsing issues, call `problems()` on your data frame for details,
e.g.:
  dat <- vroom(...)
  problems(dat) 

• readr returned 8 rows and a problems() data frame: rows 3, 5, 6 and 8, each with the column where the count went wrong and expected 4 columns, actual 5 columns (or 3, or 1), and the file name. The data frame is the audit trail, and the thing to check after every read. R also printed a warning that mentions it; in a script, run stop_for_problems(df) so the job fails.

• fread stopped: Stopped early on line 3. Expected 4 fields but found 5. Consider fill=TRUE. First discarded non-empty line: <<2,Bo,Oslo,likes coffee, no sugar>>. That is the strictest behavior of the three, and the message names the line and the content. fill = TRUE makes it continue.

• base R returned 8 rows with no error: read.csv fills short rows with NA by default and, for the long one, I did not look at how it wrapped the extra field into the next line. Treat the silence as the finding.

An unclosed quote is another case where the readers mostly warn:

Rscript lab.R unterminated ../data/unterminated.csv

[1] "warning: One or more parsing issues, call `problems()` on your data frame for details,\ne.g.:\n  dat <- vroom(...)\n  problems(dat)"
[1] "warning: Found and resolved improper quoting in first 100 rows. If the fields are not quoted (e.g. field separator does not appear within any field), try quote=\"\" to avo"

readr printed its parsing-issues warning and fread warned Found and resolved improper quoting in first 100 rows. Neither raised an error. In R a warning does not stop a script, so convert warnings that matter into errors, with options(warn = 2) or withCallingHandlers, and look at the dimensions of what came back.

The BOM

All three removed the byte order mark from the first header without being asked:

Rscript lab.R bom ../data/bom.csv

readr: id name 
fread: id name 
base R: id name 

Each printed id name. That is a pleasant difference from the Go, Java and Node readers, which leave the character in place. A file in another encoding, such as UTF-16, needs it declared: locale = locale(encoding = "UTF-16") in readr.

Speed on 2,000,000 rows

I read the 130 MB orders file with each reader and summed the amount column:

  sum = {
    cols <- cols_only(amount = col_double())
    timeit("readr read_csv", read_csv(path, show_col_types = FALSE))
    timeit("readr cols_only", read_csv(path, col_types = cols_only(amount = col_double())))
    timeit("data.table fread", fread(path))
    timeit("fread select=amount", fread(path, select = "amount"))
  },
  sumbase = timeit("base read.csv", read.csv(path)),

Rscript lab.R sum ../data/orders-2m.csv

readr read_csv         rows=2000000 total=160174646.36 time=1.25s peak_rss=469MB
readr cols_only        rows=2000000 total=160174646.36 time=0.31s peak_rss=479MB
data.table fread       rows=2000000 total=160174646.36 time=0.26s peak_rss=572MB
fread select=amount    rows=2000000 total=160174646.36 time=0.11s peak_rss=572MB

Rscript lab.R sumbase ../data/orders-2m.csv

base read.csv          rows=2000000 total=160174646.36 time=12.47s peak_rss=601MB

read_csv took 1.25 s with all columns, and 0.31 s with cols_only(amount = col_double()). fread took 0.26 s for everything and 0.11 s with select = "amount". Base read.csv took 12.47 s. All four got 160174646.36. The memory in the same runs was 469 MB for readr, 572 MB for fread and 601 MB for base R, which is the whole table in memory; selecting columns is what shrinks it, and the option names are col_select and cols_only in readr and select in fread. read.csv is not fast, and for a file this size it is the one to replace. If you do not want the data in memory at all, hand the file to DuckDB's R package and let it run the SQL.

Writing

Three writers, three sets of defaults. The same data frame, with a name containing a comma, a note with a line break, a float sum, a timestamp and a missing value:

  write = {
    d <- data.frame(id = 1:3, name = c("Smith, Jane", "Zoë", " padded"), note = c('said "hi"', "line one\nline two", NA), price = c(9.5, 0.1 + 0.2, 1234.5), when = as.POSIXct(c("2025-03-14 10:15:00", "2025-03-15 00:00:00", NA), tz = "UTC"), ok = c(TRUE, FALSE, NA))
    cat("write.csv:\n"); write.csv(d, row.names = FALSE)
    cat("write_csv:\n");  cat(format_csv(d))
    cat("fwrite:\n");     fwrite(d)
  }

Rscript lab.R write x

write.csv:
"id","name","note","price","when","ok"
1,"Smith, Jane","said ""hi""",9.5,2025-03-14 10:15:00,TRUE
2,"Zoë","line one
line two",0.3,2025-03-15,FALSE
3," padded",NA,1234.5,NA,NA
write_csv:
id,name,note,price,when,ok
1,"Smith, Jane","said ""hi""",9.5,2025-03-14T10:15:00Z,TRUE
2,Zoë,"line one
line two",0.30000000000000004,2025-03-15T00:00:00Z,FALSE
3, padded,NA,1234.5,NA,NA
fwrite:
id,name,note,price,when,ok
1,"Smith, Jane","said ""hi""",9.5,2025-03-14T10:15:00Z,TRUE
2,Zoë,"line one
line two",0.3,2025-03-15T00:00:00Z,FALSE
3, padded,,1234.5,,

• write.csv quotes every text value and the header, writes NA as the literal NA, formats the timestamp 2025-03-14 10:15:00 (and drops the time at midnight, 2025-03-15), and printed 0.3 for 0.1 + 0.2, because it uses 15 significant digits.

• write_csv (readr) quotes only when needed, writes ISO timestamps with a Z, keeps NA as NA, and printed 0.30000000000000004, the full 17 digits.

• fwrite quotes only when needed, writes ISO timestamps, writes NA as an empty field, and printed 0.3.

The two details to catch: the literal NA in two of the three, which another tool reads as the text "NA" and not as missing, and the midnight timestamps without a time in write.csv, which lose information. Set na = "" in write_csv, and choose the writer by what the next reader needs.

How to tell it worked

Check the column types after the read, with spec() or str(), and compare each identifier to the source. Run problems() and treat a non-empty result as a failure. Count the rows and compare with qsv count or DuckDB. Compare a numeric total. And read the data you wrote with a different tool; the literal NA is the thing that shows up.

What the guessing hides

readr reads the first 1,000 rows to guess types, so a column that is numeric for a thousand rows and has a code after that gets the wrong type and a problems() entry. fread samples across the file and is better at that and worse at telling you what it did. Base R guesses from the whole column and is slow. In all three, an identifier should be declared. For a big job, move the file into a database or query it in place with DuckDB, and keep R for the analysis. And write down the specification: a cols() object in the script is documentation that the next person can read.