Hashing or Masking Columns in a CSV Safely Without Leaks

Hashing an email address before sharing a file sounds safe, and for years it was the standard answer: replace [email protected] with a string of hex digits, and nobody can read it. The trouble is that a hash is a function, and a function can be run by anyone. If I know that your customers use email addresses, and I have a list of likely addresses from a breach dump, a marketing list or a dictionary of names at common domains, I can hash each one and look for matches. This article shows that attack on two lines of data, then builds the version that holds up: a keyed hash for tokens, careful normalization, and partial masking for the places a human reads the value. It is the detail behind the "mask or hash with care" point in CSV security risks that catch teams out.

Why a plain hash fails

A cryptographic hash such as SHA-256 is one-way in the sense that you cannot run it backward. It is not secret: the same input gives the same output everywhere. So the way to break it is to try inputs, and the space of email addresses, phone numbers, national IDs and dates of birth is small enough that trying them is cheap. Here is the whole attack on two addresses:

import hashlib, hmac

emails = ["[email protected]", "[email protected]"]
KEY = b"change-me-keep-in-a-vault"

def plain(e):
    return hashlib.sha256(e.encode()).hexdigest()[:16]

def keyed(e):
    return hmac.new(KEY, e.encode(), hashlib.sha256).hexdigest()[:16]

leaked = {e: plain(e) for e in emails}
print("plain sha256:", leaked)
print("keyed hmac  :", {e: keyed(e) for e in emails})

# An attacker with the plain hashes and a list of guesses
guesses = ["[email protected]", "[email protected]", "[email protected]"]
table = {plain(g): g for g in guesses}
for e, h in leaked.items():
    print("recovered:", table.get(h))

plain sha256: {'[email protected]': 'b97222ea69dadf79', '[email protected]': '25a2a844fdd9d5ab'}
keyed hmac  : {'[email protected]': 'd9c7cb00ec215078', '[email protected]': '7660d1c72b76d20c'}
recovered: [email protected]
recovered: [email protected]

The attacker never "reversed" anything. They hashed three guesses, built a lookup table and read the answers. With a real list of 100 million addresses, the table takes minutes to build on a laptop. The same applies to phone numbers (about ten billion possibilities in a national plan, a trivial space) and to any identifier with a regular shape. Truncating the hash, as I did to keep the output short, does not help either: 64 bits is plenty to identify a person in a list.

A common half-fix is to add a salt, a random string stored beside the data. A per-file salt that you publish with the file stops one precomputed table from working across files, and does nothing against an attacker who builds a fresh table with your salt. A per-row salt makes each hash different, which also destroys the property you wanted, since the same customer no longer gets the same token and joins stop working.

A keyed hash is the right tool

The right tool for a linkable token is a keyed hash, HMAC. It mixes a secret key into the computation, so without the key a guess cannot be checked. The second line of output above shows it: the keyed tokens for the same two addresses are different from the plain ones, and the attacker's lookup table (built without the key) matches nothing. Inside your own systems, the same address always produces the same token, so the same customer can be followed across rows and across files made with the same key.

That linkability is a feature and a risk. It is what lets an analyst count distinct customers from a token column. It is also what lets someone join two files you gave them if both used the same key. So decide the scope of the key: one key for an internal warehouse where linking is the point, one key per recipient for files you hand out, so that their copies cannot be matched against each other. The key lives in a secrets manager or an environment variable, never in the repository, the script's arguments or the CSV.

Normalize before you hash

Tokens are only useful if equal inputs produce equal tokens. The inputs rarely are. [email protected], [email protected] with a leading space and [email protected] are one person and three hashes. Strip surrounding whitespace and lowercase emails before hashing (the local part is technically case sensitive, in practice it is not treated that way), remove spaces and punctuation from phone numbers and convert them to one format, and write the normalization rule next to the key policy so it does not drift between jobs. Standardizing phone numbers in a CSV file shows the phone case.

A script that does both

Here is a small script that reads the customer file, writes a token column from the email with HMAC, and a masked email for display. The key arrives from the environment:

import csv, hashlib, hmac, os, sys

KEY = os.environ["MASK_KEY"].encode()      # kept outside the repo and outside the file

def token(value):
    return hmac.new(KEY, value.strip().lower().encode(), hashlib.sha256).hexdigest()[:16]

def mask_email(value):
    local, _, domain = value.partition("@")
    return f"{local[:1]}***@{domain}"

src, dst = sys.argv[1], sys.argv[2]
with open(src, newline="", encoding="utf-8") as fi, open(dst, "w", newline="", encoding="utf-8") as fo:
    reader = csv.DictReader(fi)
    fields = ["customer_token", "email_masked", "plan", "mrr"]
    writer = csv.DictWriter(fo, fieldnames=fields)
    writer.writeheader()
    for row in reader:
        writer.writerow({
            "customer_token": token(row["email"]),
            "email_masked": mask_email(row["email"]),
            "plan": row["plan"],
            "mrr": row["mrr"],
        })

MASK_KEY=change-me-keep-in-a-vault python3 mask_csv.py customers.csv masked.csv
cat masked.csv

customer_token,email_masked,plan,mrr
d9c7cb00ec215078,a***@example.test,pro,49
7660d1c72b76d20c,b***@example.test,free,0
5791e3271fc364ff,c***@example.test,pro,49
b0324f05f19dcc3c,d***@example.test,team,199

The token for Ada, d9c7cb00ec215078, matches the keyed value printed earlier for the same address and key, which is the property you want. The writer is built from a fixed field list, so a column added to the source table is not exported unless someone adds it here.

Look at the masked emails with a skeptic's eye. a***@example.test leaves the domain intact and a first letter. For a company domain, the domain names the employer. For a small domain with one customer, the first letter and the domain are nearly an identifier. Masking is for a human reading a screen or a printout where seeing the whole value would be rude or risky, and it is a weak protection for a dataset that leaves your control. If the file is shared, the token should be the only handle, and the masked value should go.

Format-preserving cases

Sometimes a downstream system needs a value that looks like the original: a ten-digit number, a postcode-shaped string. A hash does not fit, and a truncated hash may collide. Options are a format-preserving encryption scheme, which needs a library and a key, or a lookup table that maps each real value to a made-up one of the right shape and is stored securely. The table is simple and gives full control, at the price of being a secret you must protect and keep consistent across runs. For test data the table is usually the right choice, and a seeded random generator produces it repeatably.

Check what you produced

Three checks before the file goes out. Count distinct tokens against distinct inputs: they must be equal, or you have collisions or normalization gaps. Search the file for the @ sign and long digit runs, which means a column slipped through. And take two files produced a month apart with the same key and confirm that a known customer has the same token in both, because a key that changed silently breaks every join the recipient built.

When you need to get the value back

A token is a one-way trip. If a downstream process must send an email to the person, or the support team must look up a customer from a token, you need a way back, and a keyed hash does not provide one. The two choices are encryption, where the original travels as a ciphertext and the key stays with whoever may decrypt, or a token vault, a table you hold that maps each token to the real value and enforces who may query it. The vault is the common pattern for payment data and for support tools.

Whichever you use, remember the legal status: tokenized data is still personal data when you, or anyone with the key, can link it back to a person (GDPR Recital 26). For the construction of HMAC itself, the reference is RFC 2104, and Python's hmac module documents the function used above. The wider process of deciding which columns go at all is in stripping personal data from a CSV before sharing it.