XSLT and XQuery for Real Transformations Step by Step

XML is rarely the end of the road. It comes in and something else has to go out: a different XML shape, HTML, JSON, text, a number. The tools made for that are XSLT and XQuery, and between them they have been the standard answer for twenty years. They are also regarded as old and awkward, which is partly fair and mostly a matter of using the wrong version, the wrong engine or the wrong job for them.

This guide works through real transformations on files used throughout this series, with the output of each run. It covers what the languages are, which engines run which versions, the basic mechanism of templates, grouping, enrichment of a large file, JSON output, parameters, debugging, speed and testing, and ends on how to choose. Each spoke in the cluster goes deeper and is linked as it comes up.

What the two languages are

XSLT is a language for transforming one XML document into another document: XML, HTML or text. A stylesheet is itself XML, a set of template rules that match nodes in the input and say what to write. The processor walks the input and applies the rules. You describe what each kind of node becomes. It is a transformation language.

XQuery is a language for querying and building: you write an expression, usually a FLWOR (for, let, where, order by, return), that selects values from one or more documents and returns a sequence, which may be a number, a list of strings or a new XML structure. It is to XML what SQL is to tables. It is a query language that can construct output.

Both build on XPath, and both come in versions that differ more than the names suggest.

The first row is the one most people have by accident. If your runtime is PHP, Python or a shell, you have XSLT 1.0 on libxslt, and the 2.0 and 3.0 features are not available there. Everything that makes XSLT pleasant (grouping, exact arithmetic, regular expressions) arrived in 2.0, so a good deal of the reputation for awkwardness belongs to 1.0. The Home Edition of Saxon is free and runs 3.0; it needs a JVM.

The mechanism: templates

The model is covered in XSLT basics: templates, match and apply, and in one paragraph: a template has a pattern; the processor starts at the root and finds the template that fits the current node; the template's body writes output and calls apply-templates to hand child nodes to the rules; nodes with no rule go through built-in rules that descend and copy text. Two idioms cover most real work: the identity template, which copies everything, with a few overriding templates for the parts that change, and the push style, where the root template hands nodes to specific templates. The first stylesheet, with output, is in using XSLT to transform XML into another shape.

Worked transformation 1: HTML from data

A table of books as an HTML page takes one template for the root and a loop. The processor escapes < and & in values by itself, so a title with quotation marks and an ampersand in a note come out safe, and the stylesheet needs no escaping code. The full stylesheet and its output are in transforming XML to HTML with XSLT. Two cautions repeat: write a non-breaking space as  , since XML has no  , and treat copied markup (xsl:copy-of, disable-output-escaping) as untrusted.

Worked transformation 2: enrich a large file

A realistic job: take the 20,000-invoice file and produce a cleaned copy where the internal memo elements are removed, the cur attribute is renamed currency, and each invoice gets a computed total. In XSLT 3.0 the identity pattern is one line:

<xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" exclude-result-prefixes="xs">
  <xsl:output method="xml" indent="yes"/>
  <xsl:mode on-no-match="shallow-copy"/>
  <xsl:template match="memo"/>
  <xsl:template match="invoice">
    <xsl:copy>
      <xsl:attribute name="currency" select="@cur"/>
      <xsl:copy-of select="@id"/>
      <xsl:apply-templates select="node()"/>
      <total><xsl:value-of select="sum(line/(xs:decimal(amount) * xs:integer(@qty)))"/></total>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

xsl:mode on-no-match="shallow-copy" is the 3.0 replacement for the identity template: any node without a rule is copied and its children processed. The empty template for memo drops it. The invoice template rebuilds the element: it adds currency from @cur, keeps @id, processes the children and appends the total, computed with xs:decimal so the cents are exact.

real 0m2.430s
<invoice currency="EUR" id="1">
   <customer>c1</customer> ... <line sku="S244" qty="5"> ...
   <total>1016.19</total>
</invoice>

Run through Saxon in 2.43 seconds, including start-up, on a 4.4 MB input and a 6.7 MB output (indentation adds bytes). I checked the result: it contained 20,000 invoices, none with a memo (400 were removed), each with the new attribute and a total. The first invoice's total is 1016.19: five at 97.23, two at 84.87, five at 31.50 and four at 50.70, which is 486.15 + 169.74 + 157.50 + 202.80, and adds up. Every other element passed through untouched, which is the benefit of the pattern: the stylesheet mentions only what changes. In XSLT 1.0 the same job needs the explicit identity template and cannot compute the total in one expression, because sum cannot multiply two fields.

Worked transformation 3: XML to JSON

XQuery 3.1 and XSLT 3.0 both have maps and arrays and can serialize them as JSON. In XQuery:

declare option output:method 'json';
declare option output:indent 'yes';
map { 'updated': string(/library/@updated),
      'books': array { for $b in //book return map {
          'id': string($b/@id), 'available': xs:boolean($b/@available),
          'title': string($b/title), 'authors': array { $b/author ! string(.) },
          'price': xs:decimal($b/price), 'currency': string($b/price/@currency),
          'note': if ($b/note) then string($b/note) else () } } }

{
  "books": [
    { "authors": ["Ana Lee", "Bo Chen"], "price": 29.5, "available": true, "currency": "USD",
      "note": "Covers R&D", "title": "Cache Patterns", "id": "b1" },
    { "authors": ["Cy Ortiz"], "price": 44, "available": false, "currency": "USD",
      "note": null, "title": "SQL Tuning, 2nd Edition", "id": "b2" },
    { "authors": ["Di Park"], "price": 250000, "available": true, "currency": "VND",
      "note": "", "title": "Indexes \"Explained\"", "id": "b3" }
  ],
  "updated": "2026-10-01"
}

(The listing compacts the output; the real run indented each key on its own line.) You control the types here, which is the point: the flag is a JSON boolean, the price a number, the authors always an array (even for one author, because the array is built explicitly), the empty note "" and the absent note null. Compare this with the generic converters in converting XML to JSON and the attribute problem, where the shapes follow the data. Note the order of keys: authors comes first and id last, not in the order I wrote. XQuery maps are unordered collections, and the serializer chose an order. If the order matters to a reader, write the output as a string or use an ordered structure. The same output from XSLT 3.0 uses the same maps with xsl:output method="json" and ran in Saxon.

Worked transformation 4: grouping and totals

Grouping is where the versions diverge most. The same revenue-per-currency job, with the same answer in every implementation that could compute it:

EUR: 3897 invoices, 9764 lines, revenue 1,481,098.58
USD: 12138 invoices, 30245 lines, revenue 4,579,242.21
VND: 3965 invoices, 9810 lines, revenue 1,503,163.23

XSLT 1.0 needs the Muenchian key idiom and could give the counts only. It took 4.13 seconds on libxslt and 1.02 on Saxon. XSLT 2.0 for-each-group gave the counts and exact revenue in 1.50 seconds on Saxon. XQuery group by gave the same in 1.30 seconds on BaseX. A streaming Python loop did it in 0.21 seconds. The code and the comparison are in grouping records with XSLT 2 and Muenchian keys and choosing between XSLT, XQuery and a script.

Sorting, filtering and parameters

Two defaults cause most wrong results, and both are silent. The first is the sort data type: xsl:sort compares as text unless you set data-type="number", so descending prices came out as 44.00, 29.50, 250000 until the type was set (sorting and filtering XML with XSLT). In XQuery the sort key is typed, and a cast to xs:decimal gives numeric order. The second is the parameter: a value passed as an XPath expression, with --param in xsltproc or a keyword argument in lxml, reads a bare VND as a path to child elements, finds nothing and runs with an empty value, and the output says zero matches (passing parameters into an XSLT stylesheet). Use --stringparam and etree.XSLT.strparam for text.

Debugging

When the output is empty or the whole file's text, the templates did not match. The five causes I found are in debugging an XSLT transform that outputs nothing. The instrument that finds most of them is a low-priority catch-all that reports what it sees:

<xsl:template match="*" priority="-10">
  <xsl:message>unhandled <xsl:value-of select="name()"/> in <xsl:value-of select="namespace-uri()"/></xsl:message>
  <xsl:apply-templates/>
</xsl:template>

On a namespaced Atom feed with a stylesheet written for no namespace it printed unhandled feed in http://www.w3.org/2005/Atom, which is the whole diagnosis in one line. Beyond that: count what a select finds with xsl:message, print name() and namespace-uri() of the current node, and shrink the input until it fits on a screen. In lxml, messages go to the stylesheet's error_log and not to the terminal (running XSLT in Python with lxml).

Running it

The commands, for reference:

xsltproc --stringparam currency VND books.xsl library.xml
java -cp "lib/*" net.sf.saxon.Transform -s:invoices.xml -xsl:enrich.xsl -o:out.xml
java -cp "lib/*" org.basex.BaseX -i library.xml query.xq

The first line runs XSLT 1.0, the second XSLT 2.0 or 3.0 on Saxon and the third an XQuery on BaseX. The Java jars come from Maven Central (net.sf.saxon:Saxon-HE:12.5 and org.basex:basex:11.6 were the coordinates I used). In Python, etree.XSLT runs 1.0 only; for 2.0 and 3.0 call Saxon as a subprocess. In .NET, XslCompiledTransform implements 1.0, and Saxon has a .NET build. Browsers run 1.0 through XSLTProcessor. The command-line details for xsltproc are in running XSLT from the command line.

Speed

Parsing XML is fast and the cost of a transform is in the idioms. All five of the grouping implementations finished a 4.4 MB file in under five seconds, and the differences were fourfold between libxslt and Saxon on the same stylesheet, and tenfold between Saxon and a Python loop on a tree already in memory. For large files, the tree is the issue: XSLT and XQuery in their basic forms build the input tree in memory, 15 to 40 times the file size. XSLT 3.0 can stream, but the streaming mode needs a licensed Saxon edition, and I did not test it. For files that must not be loaded whole, a streaming script does the aggregation and hands small documents to a stylesheet for presentation.

What XSLT 3.0 adds beyond 1.0

The jump from 1.0 to 3.0 is large enough to deserve a look at the new instructions, because they remove the workarounds that made the older language tedious. I ran a stylesheet on the invoice file that uses four of them:

<xsl:iterate select="invoice[position() le 4]">
  <xsl:param name="acc" select="xs:decimal(0)"/>
  <xsl:variable name="t" select="sum(line/(xs:decimal(amount) * xs:integer(@qty)))"/>
  <xsl:value-of select="@id"/>=<xsl:value-of select="$acc + $t"/><xsl:text> </xsl:text>
  <xsl:next-iteration><xsl:with-param name="acc" select="$acc + $t"/></xsl:next-iteration>
</xsl:iterate>
<xsl:for-each-group select="invoice[position() le 12]" group-adjacent="@cur"> ... </xsl:for-each-group>
<xsl:try>
  <xsl:value-of select="xs:integer('2x')"/>
  <xsl:catch><xsl:value-of select="$err:description"/></xsl:catch>
</xsl:try>

running totals: 1=1016.19 2=1114.74 3=1441.8 4=1555.1
runs of the same currency in the first 12: EURx1 USDx1 EURx3 USDx1 EURx1 USDx1 EURx1 USDx1 EURx1 USDx1
caught: Cannot convert string "2x" to an integer
text value template: hello Ada, total invoices 20000

**xsl:iterate** is a loop that carries state from one item to the next through parameters. The running total acc starts at zero and passes acc + t into each next iteration, which gives 1016.19, then 1114.74, 1441.8 and 1555.1: exact decimals, accumulated. In 1.0 this is a recursive template with all the ceremony that implies; here it is a loop, and Saxon can run it without keeping earlier items.

**group-adjacent** groups consecutive items with the same key. In the first twelve invoices, the currency changes often (the file is not sorted by currency), and the output lists runs: one EUR, one USD, three EUR, and so on. This is the instruction for log-like data, where the runs matter, and it has no cheap equivalent in 1.0.

**xsl:try and xsl:catch** turn a dynamic error into a value. The cast of '2x' to an integer raised an error, the catch block ran, and the message was available as $err:description. Without it, the whole transform stops on the first bad value. With it, a stylesheet can log the problem and continue, which is how you process a feed with a few dirty records.

Text value templates (expand-text="yes") allow {$n} inside text. The last line shows a variable and a count() expression interpolated in a literal, which removes most xsl:value-of noise.

All four need a 3.0 processor. A 1.0 engine rejects them or ignores them, which is the argument for stating the version and the engine in the repository.

The shared expression language

XSLT 2.0 and 3.0 and XQuery share XPath 3.1, which is a good deal more than XPath 1.0. A query I ran on the library file shows the range:

let $books := //book
return (
  'distinct currencies: ' || string-join(distinct-values($books/price/@currency), ','),
  'all have a title: ' || (every $b in $books satisfies exists($b/title)),
  'tokenize: ' || string-join(tokenize('Cache Patterns for SQL', '\s+'), '|'),
  'replace: ' || replace('SQL Tuning, 2nd Edition', '(\d+)(st|nd|rd|th)', '#$1'),
  'date math: ' || (xs:date('2026-10-27') + xs:dayTimeDuration('P30D')),
  'sum with format: ' || format-number(sum($books/price[@currency = 'USD'] ! xs:decimal(.)), '#,##0.00')
)

distinct currencies: USD,VND
all have a title: true
tokenize: Cache|Patterns|for|SQL
replace: SQL Tuning, #2 Edition
date math: 2026-11-26
sum with format: 73.50

Every line here is impossible or painful in XPath 1.0: distinct-values, quantified expressions (every, some), regular expressions with tokenize and replace, date arithmetic with typed durations (27 October plus 30 days is 26 November), and format-number over an exact decimal sum. One caveat from the same run: XPath regular expressions follow XML Schema's dialect, which has no \b word boundary. My first pattern ^[A-Z]{3}\b was rejected with FORX0002 Invalid regular expression, and I rewrote it as ^[A-Z]{3} with a space. If you port patterns from a language with Perl-style regular expressions, test each one.

Namespaces in the modern languages

The namespace trap exists in all of them, with better tools. In XSLT 2.0 and later, xpath-default-namespace="http://www.w3.org/2005/Atom" on the stylesheet element makes unprefixed element names in paths mean the Atom namespace, so the namespaced feed from the debugging article needs no prefixes in its paths. XQuery has declare default element namespace "..." in the prolog for the same purpose. Both solve the zero-results problem at the root, and both can hide the namespace from a reader, so comment them. In XSLT 1.0 there is no such declaration, and every path needs a prefix.

A cross-engine check you can run in CI

The cheapest guard against engine surprises is to run the same job on every engine you rely on and compare normalized output. I wrote a short shell script that runs the grouping on xsltproc (the 1.0 stylesheet), Saxon (the 2.0 stylesheet), BaseX (the query) and Python, strips each output down to the currency and the invoice count, sorts it and compares:

norm() { sed -E 's/^([A-Z]+): ([0-9]+) invoices.*/\1 \2/' | sort; }
xsltproc group10.xsl invoices.xml | norm > out_xsltproc.txt
java -cp "lib/*" net.sf.saxon.Transform -s:invoices.xml -xsl:group20.xsl | norm > out_saxon.txt
java -cp "lib/*" org.basex.BaseX q4.xq | norm > out_basex.txt
python3 count.py | norm > out_python.txt
for f in out_saxon.txt out_basex.txt out_python.txt; do diff -q out_xsltproc.txt $f && echo "xsltproc == $f"; done

xsltproc == out_saxon.txt
xsltproc == out_basex.txt
xsltproc == out_python.txt
EUR 3897
USD 12138
VND 3965

All four agree on the count per currency, which is the property I want from a regression test. The normalization step is the design: each tool prints in its own style (the 2.0 stylesheet and the query also print revenue, the 1.0 one does not), so the script compares only what all four produce. When two outputs diverge, diff shows the lines, and the divergence is a bug in one of them or a difference in what they mean by the same word. The same structure works for a deployment where the production engine differs from the development one: run the fixtures through both and fail the build on any difference.

Add the awkward fixtures to it. For this job, a document with a currency that appears once, one with no invoices, one whose cur attribute is missing on a single invoice (does it become its own group, or an error?) and one with a different case (usd against USD). The point of a cross-engine test is to find the cases where engines disagree, and the cases that make engines disagree are rare values, not the typical ones.

Habits that keep stylesheets maintainable

A few practices that cost nothing and pay off the first time someone else reads the file.

One purpose per stylesheet. A stylesheet that renames elements, computes totals and produces HTML is three stylesheets. Chain them with xsltproc and a pipe, or import them, so each is testable alone.

Comment the clever parts. The Muenchian idiom, a generate-id() comparison and any use of preceding-sibling for a distinct count deserve a line saying what they do.

Name the version and the engine at the top, in a comment and in the version attribute.

Prefer explicit over implicit. select expressions that say book when they mean child::book work, and a reader who knows XPath well enough to catch the difference between //book and book does not need the longer form. What a reader needs is a stylesheet where the context node is easy to track: use short templates, name variables for what they hold, and avoid deep for-each nesting.

Keep data out of stylesheets. A list of codes and labels belongs in a small XML file read with document(), not as forty xsl:when branches.

Keep presentation out of data. If the stylesheet needs to know that 250000 is dong and should be shown with a different format, put the currency list in the data or a parameter, not in the template logic.

Reading a failure from each engine

The three engines report problems differently, and knowing the shape of each saves time. xsltproc prints a line for a stylesheet it cannot compile, with the file and the line, and exits non-zero. Saxon prints an error code from the W3C error list (XTSE0010 for a static error in a stylesheet, XTDE codes for dynamic errors, FORG0001 for a failed cast) with the line number and a message; the codes are searchable and consistent across processors. BaseX prints Stopped at followed by the file and position, and an error code (XPST0003 for a syntax error, FORX0002 for a bad regular expression, as in the run above). Search the error code, not the English message; the codes are stable and the wording is not.

Dynamic errors deserve a plan. An XSLT 1.0 transform converts a bad number to NaN and goes on, so a bad value shows up as a wrong total or a missing row. A 2.0 or 3.0 transform raises an error and stops, unless you wrap the risky step in xsl:try. XQuery raises as well, and the try ... catch expression handles it. The stricter behavior is better for data, because a stop is visible; and it makes the first run on real data more eventful. Run new stylesheets on a sample of the real feed before you schedule them, with the awkward values present.

Choosing an engine and a version

The practical decision is usually made for you by the runtime, and it is worth being explicit about it. If the transform runs inside PHP, Python or Ruby, you have libxslt and XSLT 1.0, and the sensible move is to keep stylesheets small and presentational, and to do arithmetic, grouping and text processing in the host language. If you can run a JVM process, Saxon gives you XSLT 3.0 and XQuery 3.1, and a stylesheet can be written in the modern style. If you are building a service that queries a collection of documents, an XML database (BaseX is a free one) gives XQuery with indexes.

Whichever you pick, declare the version in the stylesheet (version="1.0", "2.0", "3.0") and keep it consistent with the engine you deploy. A stylesheet written for 3.0 that lands on libxslt fails at the first function it does not know, or, worse, runs in forwards-compatible mode and silently does something else. Put the engine and its version in the README next to the stylesheet.

Migrating from 1.0

Teams with years of 1.0 stylesheets often ask whether to migrate. My answer is to migrate selectively. A stylesheet that restructures documents and works should be left alone: 1.0 is stable and fast enough. A stylesheet that has grown helper templates to simulate grouping, string splitting or number formatting is the candidate, because each helper is a few lines in 2.0 or 3.0. The migration is mechanical for most of it: 2.0 is largely backward compatible, and the biggest surprises are the stricter typing (a comparison of a string with a number that 1.0 coerced is an error in 2.0) and the different handling of sequences. Run the old and new stylesheets on the same fixtures, compare canonical output, and switch when they agree.

Testing transforms

A transform is code, and the tests are small. Keep an input file with the awkward cases (a group of one, an empty element, an absent element, an ampersand, a namespace, an empty document) and its expected output beside it, and compare the actual output in CI: for XML, compare canonical forms so that attribute order and indentation do not matter (comparing two XML documents for differences); for text, compare bytes. Run the same fixtures through every engine you deploy to. A stylesheet that works on Saxon and fails on libxslt because of a 2.0 function is found by that test, not by the first user.

Security

A stylesheet is a program that can read files with document() and, in some engines, run extension code or fetch URLs. Never run a stylesheet from an untrusted source against your data or your server. For stylesheets you do write, keep input documents safe (no DTDs or external entities, size limits; see XML security problems to close before production), disable network and file access where the engine allows it (XSLTAccessControl in lxml blocked a document() call with read rights for file denied), and avoid disable-output-escaping on values from outside.

Where to read next

The spokes of this cluster: XSLT basics: templates, match and apply, grouping records with XSLT 2 and Muenchian keys, sorting and filtering XML with XSLT, passing parameters into an XSLT stylesheet, debugging an XSLT transform that outputs nothing, running XSLT in Python with lxml, XQuery basics with BaseX and choosing between XSLT, XQuery and a script. The earlier conversion articles: using XSLT to transform XML into another shape and transforming XML to HTML with XSLT.

The specifications: XSLT 3.0, XSLT 1.0, XQuery 3.1 and XPath 3.1. The engines: Saxonica documentation, BaseX documentation and the libxslt project. Test an XPath in the XPath tester.

When a transformation language is the wrong tool

The job is mostly logic. Branches, loops with state, calls to services and error handling per record belong in a program.

The data is large and the job is aggregation. A streaming script does it in a fraction of the time and memory.

Nobody on the team reads these languages. A transform that one person can maintain is a risk. Put presentation in a stylesheet if you must, and keep logic in code the team knows.

The input is not XML. Convert it first, or use the tools of its own format.

A library already does it. For standard conversions (office formats, feeds, SVG), a purpose-built library will have solved the edge cases.