Handy Command Line Tools for XML Work in Daily Practice

A terminal is the fastest place to find out what is in an XML file. You do not need an editor, a project or a script. A handful of programs cover almost everything: check that a file is well-formed, make it readable, pull a value out, count something, edit a field, validate against a schema, transform it into something else. They are small, they are installed or one package away, and they compose with the rest of the shell.

This guide is a map of those tools and a cookbook for the jobs where I reach for them. Every command was run, and the output shown is what it printed. The spokes in this cluster take each tool or job further, and they are linked as they come up.

The toolbox

The first three are the core, and they share an engine: xmllint and xsltproc are the command-line faces of libxml2 and libxslt, and xmlstarlet is built on both. That means the same XPath 1.0 semantics and the same parsing rules and the same limits across all three, including the namespace behavior and the depth and text-size guards. If a path works in one, it works in the others.

A note on installing. xmllint and xsltproc are in the libxml2-utils and xsltproc packages on Debian-family systems and ship with macOS (check the version with xmllint --version, since the system copy can lag). xmlstarlet is apt install xmlstarlet or brew install xmlstarlet; in a minimal container it is usually missing, and in my test container one apt-get install supplied it. Windows users get all of these inside WSL.

Check: is it XML at all

xmllint --noout file.xml

This is the first command to run on any file, and its exit code is the interface. I ran it on a good file, a broken one and a missing one, and an XPath that selects nothing:

Scripts should test the code, not the text of the message. Notice that code 1 covers both a broken file and a missing one, so a script that needs to tell them apart tests for the file first. And code 10 is easy to miss: an XPath that matches nothing is not an error in the XPath sense, and a set -e script stops on it. In a loop that counts matches, use count(...), which returns a number (possibly 0) with exit 0, instead of selecting the nodes.

For big files, the check has a streaming mode that does not build a tree. Timings on the 36.6 MB, 200,000-order file:

A third of a second to check a file this size means the shell is rarely the bottleneck. The limit is memory: the tree modes build the whole document, which for this file is hundreds of megabytes, and --stream does not.

Read: format and look

xmllint --format oneline.xml
xmllint --noblanks --format indented.xml
XMLLINT_INDENT='    ' xmllint --format oneline.xml
curl -s https://example.test/feed.xml | xmllint --format - | less

Formatting is the cheapest way to make a minified response readable, and with --noblanks it gives a canonical layout that makes diffs small. Details and the caveats on mixed content and signed files are in formatting XML with xmllint for readable diffs. The dash reads from standard input, which is how you format a response without saving it.

To explore interactively there is a shell: xmllint --shell file.xml accepts cd /library/book[2], ls, cat title and quit. It is crude and useful for a first walk through a structure you do not know. For the structure as a list, xmlstarlet el -u file.xml prints each distinct element path once, which for the 20,000-invoice file was seven lines.

Query: one value, many values

For one value, xmllint --xpath:

xmllint --xpath 'string(//book[1]/note)' library.xml
xmllint --xpath 'count(//invoice[@cur="USD"]/line)' invoices.xml

The string() wrapper is the habit to build. Without it, xmllint prints the XML of the selected nodes, tags and escapes included (<note>Covers R&amp;D</note>), which is rarely what a script wants.

For many values, xmlstarlet sel with a template prints one line per record:

xmlstarlet sel -T -t -m '//book' -v '@id' -o ',' -v 'price' -o ',' -v 'price/@currency' -n library.xml

b1,29.50,USD
b2,44.00,USD
b3,250000,VND

Without the -T, xmlstarlet escapes its text output and prints Covers R&D. And it does not quote, so a title containing a comma breaks a naive CSV; use tabs, or leave CSV to a script. The full template language is in extracting values with xmlstarlet.

Counting deserves its own warning, because the obvious tool is wrong. grep -c '<line' invoices.xml printed 20000, the number of physical lines with a match, for a file with 49,819 line elements, and on a one-line file with three elements grep -c printed 1. grep -o ... | wc -l and the XPath count both gave 49,819. Count with XPath or a parser (counting nodes in XML from the shell).

Edit: with a parser, not with sed

xmlstarlet ed -u '//book[@id="b2"]/price' -v '39.00' library.xml        # preview on stdout
xmlstarlet ed -L -u '//book[@id="b2"]/price' -v '39.00' library.xml     # in place

An edit through the parser changed exactly the three things I asked for in a three-edit command, kept a comment, and escaped a & b < c correctly when it was a new value. The sed equivalent worked for a literal on one line and would have inserted the raw characters otherwise. The habits that make scripted edits safe (write to a new file, validate, check that the XPath matched something because ed exits 0 even when it selected nothing) are in editing XML files in place from a script.

Transform: xsltproc and the yq family

xsltproc applies a stylesheet: xsltproc --stringparam currency VND books.xsl library.xml. It exits 4 when the stylesheet cannot be loaded, and its --timing option splits the run into parse, apply and save phases. The stylesheet language is XSLT 1.0, which has no grouping or regular expressions; see running XSLT from the command line and using XSLT to transform XML into another shape. Use it for markup output (XML, HTML) and be wary of producing CSV with it, since a stylesheet that wrapped titles in quotes without doubling the inner ones wrote an invalid row.

For jq-style pipelines there are two programs named yq. The Python one provides xq, and printed the first book with @id and #text keys. The Go one, run as yq -p=xml -o=json, printed +@id and +content. The structure is the same and the key names are not, so know which you have (yq --version distinguishes them) before you copy a filter. Both turn a repeated element into an array and a single one into a plain value (using yq to read XML).

Combine: merge and split

Files go together and come apart by structure, not by lines:

• cat a.xml b.xml produced two declarations and two roots, and xmllint rejected it with XML declaration allowed only at the start of the document.

• xmllint --xinclude merged three parts into one root through an include driver file, adding an xml:base attribute to each copied element unless --nofixup-base-uris is given.

• split -l 50000 cut a 36.6 MB file into five chunks, none of them valid. A streaming script split the same file into eight valid parts of 25,000 orders in 1.5 seconds at 21 MB, and the part counts summed to 200,000.

See merging several XML files into one and splitting a big XML file by repeating element for the commands and the reconciliation checks.

The dangerous options

Two options in this toolbox help a little and mislead a lot.

xmllint --recover parses a broken file and prints what it can. On <r><a>1</a><b>2</c><d>4</d></r> it reported the mismatch and printed <r><a>1</a><b>2</b><d>4</d></r>, treating </c> as if it were </b>. That is a guess, and often the right one; for data it is a way to load a corrupted value without any marker that it was corrupted. Use it to look, and never in a pipeline that loads.

xmllint --noent substitutes entity references, and with an external entity in the file it reads the target (what an XML external entity attack is). Do not run it on files you did not write. Also pass --nonet to every xmllint and xsltproc call on input from outside, so nothing is fetched from the network; the XML security cluster starts at XML security problems to close before production.

Recipes

A few combinations that I use repeatedly.

Fetch, check, look.

curl -fsS "$URL" -o feed.xml && xmllint --noout --nonet feed.xml && xmllint --format feed.xml | head -40

-f makes curl fail on an HTTP error instead of saving an error page, which you would otherwise discover as a parse error on an HTML body.

Profile an unknown file.

xmlstarlet el -u file.xml          # distinct paths
xmllint --xpath 'count(//*)' file.xml

The first command printed seven paths for the invoices file. The second counts every element, a quick measure of size and a number to reconcile later.

A gate in a pipeline.

xmllint --noout --nonet --schema shop.xsd "$f"; case $? in 0) ... ;; 1) echo not well-formed ;; 3) echo schema ;; esac

Exit codes distinguish the two kinds of rejection (validating XML against XSD with xmllint).

Preview and apply an edit. Run xmlstarlet ed without -L and diff against the original; when the diff is right, rerun with -L on a copy and move it into place.

Many files. A shell loop that starts one process per file took 3.2 seconds for 2,000 small files, parallel xargs -P8 0.69 seconds, and a single Python process 0.27 seconds. Past a few thousand files, move the loop into a script (extracting a field from thousands of XML files).

Namespaces in the shell

The namespace trap is the same in every tool, and each has its own way around it. In xmllint --xpath there is no way to bind a prefix, so use local-name(): //*[local-name()="url"], which matches any namespace and so can overcount. In xmlstarlet, -N s=URI binds a prefix, and without it the sitemap's //url printed nothing while //s:url printed the three URLs. In xsltproc, declare the prefix in the stylesheet. In the yq tools, the prefix stays in the key name. When a count is zero on a file you know has data, check the root element for xmlns first (how XML namespaces work).

Choosing between them

My rules of thumb:

• A yes-or-no or a single number: xmllint.

• A column of values, a list of paths, or an edit: xmlstarlet.

• A reshaping that is mostly structural: xsltproc, or a script.

• **A pipeline with other formats and jq knowledge:** a yq.

• Anything with quoting, types, error handling, many files or a big file you must keep in memory: Python and lxml.

The line between the shell and a script is where the first quoting rule or the first error-handling branch appears. A one-liner that needs a second line of logic has become a script, and a script gives you tests and a place to put the reconciliation.

Reading the error messages

The parser errors from these tools all come from libxml2 and follow one format: the file, the line, parser error, the message, then the offending line with a caret under the column. Nine small files, each broken in a different way, produced the messages you will meet most:

A few notes on reading them. The first message in a file is the real one; the ones after it are often consequences, like the Premature end of data that followed the mismatch in the first row. The caret points at where the parser noticed, which is usually at or after the actual mistake: in <a>x & y</a> it points just after the ampersand, which is the character at fault, and in the mismatch it points past the closing tag. The line number is exact for a pretty-printed file and useless for a file on one line, which is another reason to format first when the file parses far enough, or to pipe it through fold or tr '>' '>\n' to get lines to look at.

The messages also map to causes. xmlParseEntityRef: no name is a raw ampersand, the commonest defect in hand-edited or template-generated XML. Entity 'nbsp' not defined is HTML habits in an XML file: XML defines only five named entities (lt, gt, amp, quot, apos), so a non-breaking space must be written as a number reference. Extra content at the end of the document is a second root, usually from concatenating files. Document is empty is a zero-byte file, often the result of a failed download. The byte order mark case is the sneaky one, because the characters before the declaration are invisible in most editors. Each of these has an article of its own, starting with fixing XML parse errors fast and undefined entity errors in XML.

Compressed input and pipes

Large XML is usually compressed, and the shell handles that well because the tools read standard input. Streaming mode works on a pipe:

gzip -c invoices.xml > invoices.xml.gz
zcat invoices.xml.gz | xmllint --noout --stream -

(no output; exit 0)
real	0m0.034s

The 4.4 MB file compressed to 513 KB, a ratio of about 8.6 to 1, and the streaming well-formedness check on the decompressed pipe took 0.03 seconds. The dash tells xmllint to read standard input. The same pattern works for bzcat, xzcat and curl -s ... | gunzip. The point is that you never need to unpack a multi-gigabyte archive onto disk to check it or to profile it. Remember that the tree-building modes (--xpath, --format, --schema without --stream) read the whole decompressed document into memory, so for a large compressed file use --stream where it applies, or a streaming script.

A Makefile and a hook

Commands you run twice belong in a file. A two-target Makefile checks and format-checks the XML files in a directory:

XMLS := $(wildcard data/*.xml)

.PHONY: check fmt-check
check:
	@for f in $(XMLS); do xmllint --noout --nonet "$$f" || exit 1; done
	@echo "checked $(words $(XMLS)) files"

fmt-check:
	@for f in $(XMLS); do xmllint --noblanks --format "$$f" | diff -q - "$$f" >/dev/null || { echo "needs formatting: $$f"; exit 1; }; done
	@echo "all formatted"

checked 2 files
all formatted

With two files in data/ (one as received, one run through xmllint --noblanks --format first), make check passed both, and make fmt-check reported that both were already formatted. The second target is the pre-commit idea: it fails when a file's formatted form differs from the file, so the repository settles on one layout and diffs stay small (formatting XML with xmllint for readable diffs). Hook the targets into your CI, or into a Git pre-commit hook that runs make check. The loop exits on the first failure with the file's name already printed by xmllint, which is the information a developer needs.

Two details in the Makefile are worth copying. The $$f doubles the dollar sign so Make passes a single $ to the shell. And --nonet is there even though these are local files, because a build step that parses XML and could fetch a DTD from the network is a build that sometimes fails offline and sometimes does something you did not intend.

What these tools do not do well

I have been positive about the toolbox, so here are the limits, collected.

They parse XML 1.0 and XPath 1.0. That is libxml2's world: no max, min, regular expressions or grouping in XPath, no XSLT 2.0 or 3.0, no XQuery. For those, you need Saxon or BaseX, which is a different install and is covered in XSLT and XQuery for real transformations.

They do not understand your data. They check syntax and, with a schema, structure. A price of -5 is fine to all of them.

Error handling is exit codes. Good enough for a gate, not for a report of what failed across a thousand files.

Output is text. There are no types. A number that must be a decimal is a string until you convert it.

They read the whole tree in most modes. --stream for well-formedness and validation is the exception; for aggregation use a script.

Knowing where the toolbox ends is part of using it. Most days it covers everything I need. On the day it does not, the next step is a twenty-line script, and the commands above are the prototypes of what the script should do.

A worked session: a sitemap in four commands

To show how the pieces combine, here is an audit of the generated 5,000-URL sitemap from analyzing sitemap data with SQL, done entirely in the shell. The namespace is bound once in a variable, since a sitemap is in a default namespace and every path needs it.

NS="s=http://www.sitemaps.org/schemas/sitemap/0.9"
xmlstarlet sel -N $NS -t -v "count(//s:url)" -n sitemap_big.xml
xmlstarlet sel -N $NS -t -v "count(//s:url[not(s:lastmod)])" -n sitemap_big.xml
xmlstarlet sel -T -N $NS -t -m "//s:url" -v "s:loc" -n sitemap_big.xml | cut -d/ -f4 | sort | uniq -c | sort -rn
xmlstarlet sel -T -N $NS -t -m "//s:url[s:lastmod]" -v "s:lastmod" -o " " -v "s:loc" -n sitemap_big.xml | sort -r | head -3

5000
715
   1292 guides
   1256 library
   1231 topics
   1221 tools
2026-12-28 https://example.test/topics/page-4469
2026-12-28 https://example.test/topics/page-3394
2026-12-28 https://example.test/topics/page-3299

The first two commands give the control totals: 5,000 URLs, 715 of them without a lastmod. The third extracts every address as plain text (-T, so an ampersand in a URL stays an ampersand), takes the fourth slash-separated field, which is the first path segment, and counts the sections with sort | uniq -c. The fourth lists the newest dates, and finds that the most recent lastmod is 28 December 2026, which is after the day this ran, so some dates are in the future. All of the figures agree with the DuckDB version of the same audit. The extraction of 5,000 addresses took about 0.01 seconds.

Notice how the work divides. XML tools did the parts that need XML (the namespace, the selection, the escaping) and the classic text tools did the parts that are text (cut, sort, count). Each stage prints something you can inspect, which makes the pipeline easy to debug: run the first stage alone, then add one more. When a pipeline like this grows beyond four stages, or you want to run it again next week, move it into a script. It will be a short one, because you have already worked out the steps.

Habits that prevent mistakes

Some practices that save me from the same few errors.

Look before you change. Format a copy and read the top. Count the root's children. Only then edit or extract. Ninety percent of the surprises (a namespace, an unexpected encoding, a second root) show up in the first ten seconds.

Quote everything. XPath expressions in single quotes, variables in double quotes, file names in double quotes. An unquoted $f with a space in the name is two arguments, and an unquoted * in an XPath is a glob.

Test the empty case. Run the extraction on a file where the thing you want is missing, to see whether the command prints nothing, an empty line, null or an error. Then you know what a missing value looks like downstream.

Keep the original. Work on copies, and do not format, edit or recover the only copy of anything. A file that a partner sent is evidence of what they sent.

Name the version. When a result matters, record the tool versions (xmllint --version, xmlstarlet --version) in the script's header or the commit message. Behavior, especially around entities and limits, changes between releases.

Write the number you expect. Before running a count, say what you think it will be. A wrong guess teaches you something about the file, and a right one is a small confirmation that the command does what you think.

Portability and versions

The commands in this guide ran on Linux with GNU tools. On other systems, three things differ. macOS ships xmllint and xsltproc as part of the system, and their versions can be older than the ones in a current Linux distribution, which matters for the error messages and for the limits described in the security cluster; check xmllint --version and do not assume the output matches what is printed here. macOS also ships BSD versions of sed, grep, split and sort, whose options differ from GNU's: sed -i needs an argument on the BSD version, and split -l works the same way but names its output files differently on some versions. The shell-level advice (parse, do not pattern match) holds on all of them. On Windows, run the toolbox inside WSL, where it behaves like the Linux examples.

Pin the tools in anything that other people run. A container image with libxml2-utils, xmlstarlet and xsltproc installed at known versions is the cheapest way to make a script behave the same on every machine, and it doubles as documentation of what the script needs. If you cannot use a container, print the versions at the start of a long run:

xmllint --version 2>&1 | head -1
xmlstarlet --version | head -1
xsltproc --version | head -1

The output of these three lines belongs in the log of any job whose result someone will question later. Behavior that can differ between libxml2 releases includes the handling of external entities, the limits on depth and text size, and the amplification guard, so the version is a fact about the result.

Where to read next

The spokes: formatting XML with xmllint, extracting values with xmlstarlet, editing XML files in place from a script, using yq to read XML, counting nodes in XML from the shell, merging several XML files into one and splitting a big XML file by repeating element. Related: validating XML against an XSD schema, analysis on XML data when nobody has converted it yet and pretty-printing XML to spot structural errors.

The references: the xmllint manual, the xmlstarlet user's guide, the xsltproc manual, the XPath 1.0 specification and the documentation of the Go yq XML support. Test an expression in the XPath tester.

When the shell is the wrong place for XML work

You need to handle failure. A job that must retry, report which of 10,000 files failed and why, and keep going wants a script with exceptions, not a pipeline whose first error stops everything or is ignored.

You need typed values. Money as a decimal, dates as dates, booleans from a mapping. The shell sees text. Use Python, and parse the numbers properly.

The output must be valid CSV or JSON. None of these tools quotes text for CSV. Use a library that does.

The file is bigger than memory and the job needs more than a stream. xmllint --stream checks and iterparse aggregates; anything needing random access goes to a database.

The work will be repeated by other people. A command in your shell history is not documentation. Put it in a script in the repository, with a sample input and the expected output.