Sorting and Filtering XML With XSLT and the Text Sort Trap

Sorting and filtering are the two things you do to a list of records before you show it, and in XSLT each has one correct place. Filtering is an XPath predicate on the nodes you select. Sorting is an xsl:sort element inside the instruction that processes them. The only common bug is the default data type of the sort, and it is a quiet one.

Filtering: put it in the select

Select the nodes you want, and nothing else sees the rest:

<xsl:apply-templates select="book[price/@currency='USD']"/>

or, in a loop:

<xsl:for-each select="book[price/@currency='USD' and number(price) &gt; 30]">

A predicate in the select has two effects. It removes the unwanted nodes from the list the instruction works on. And it changes what position() and last() mean inside the loop: they count within the filtered list, so a numbered list of the USD books runs 1, 2, 3 without gaps. If you instead loop over all books and use xsl:if to skip some, the positions still count the skipped ones, and your numbering has holes. Filter in the select; use xsl:if and xsl:choose for decisions about how to output a node you are keeping.

Note the escaped > in the predicate. A stylesheet is an XML document, so < and & must be escaped in attribute values (<, &), and > is allowed raw but conventionally escaped for symmetry. number(price) > 30 converts the text to a number before comparing, because XPath 1.0 compares a string value with a number by converting the string, but two string values compare as text. For prices, always convert.

Sorting

<xsl:template match="/library">
  <xsl:for-each select="book">
    <xsl:sort select="price" data-type="{$mode}" order="descending"/>
    <xsl:value-of select="price"/><xsl:text> </xsl:text><xsl:value-of select="title"/><xsl:text>&#10;</xsl:text>
  </xsl:for-each>
</xsl:template>

xsl:sort goes first inside xsl:for-each (or inside xsl:apply-templates), and its attributes say what to sort by (select), how to compare (data-type: text or number), which direction (order: ascending or descending), and for text, in which language and with what case rule. I made data-type a parameter (with an attribute value template) to run the same stylesheet both ways:

xsltproc --stringparam mode text sort.xsl library.xml
xsltproc --stringparam mode number sort.xsl library.xml

44.00 SQL Tuning, 2nd Edition
29.50 Cache Patterns
250000 Indexes "Explained"
--
250000 Indexes "Explained"
44.00 SQL Tuning, 2nd Edition
29.50 Cache Patterns

The default is text. Sorted descending as text, the prices come out as 44.00, 29.50, 250000, because the comparison goes character by character and 4 is greater than 2, and 2 followed by 5 is below 2 followed by 9. The most expensive book, at 250000, is last. Sorted as numbers, it is first. (The 250000 is in Vietnamese dong, a different currency, so a real report would filter or convert first; the point is the ordering rule.) On this sample the text order looks almost reasonable, and the bug appears when the values have different lengths: 9.99 sorts above 100.00 as text. The fix is the one attribute. Whenever the key is a number, a date in a sortable form or an integer, say data-type="number" (for ISO dates, text sorting works and numbers do not, so keep text).

Several keys, and missing keys

Several xsl:sort elements in a row define a primary, then secondary key:

<xsl:sort select="price/@currency"/>
<xsl:sort select="price" data-type="number" order="descending"/>
<xsl:sort select="title"/>

This sorts by currency, then by price descending within each currency, then by title. The order of the elements is the order of precedence.

A key that is missing or not a number deserves a plan. With data-type="number", text that is not a number converts to NaN, and an absent element yields an empty node-set that also converts to NaN. I tested a list of values 10, abc, an absent element, 2 and 100: in both libxslt and Saxon the output was [abc][][2][10][100] ascending and [100][10][2][abc][] descending, so the non-numeric and missing keys sorted as the smallest values and came first in an ascending report. That is a behavior to know and not one to build on, because a report that shows broken records at the top, or buries them at the bottom, hides the real problem. The practical rule is to filter out records without a usable key with a predicate such as book[number(price) = number(price)] (NaN is the one value that is not equal to itself), sort them separately if you must show them, and count them.

Case and language

XSLT 1.0 text sorting uses a processor-dependent collation, which on libxslt is code point order unless you give lang. Uppercase letters sort before lowercase ones, so Zebra comes before apple. For a case-insensitive sort, map the key to one case with translate:

<xsl:sort select="translate(title, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz')"/>

because 1.0 has no lower-case. It handles ASCII letters only, so accented capitals need adding to both alphabets. XSLT 2.0 has lower-case() and a collation attribute with real locale rules, and Saxon supports them. For a user-facing list in a language with its own alphabet order (German, Swedish, Spanish), sort with a collation in 2.0, or in the host language, which has locale libraries. A code point sort puts Ångström after Zebra.

Sorting is stable, and sorts a list, not the tree

xsl:sort is stable: items with equal keys keep their input order, so a secondary order you want to preserve can come from the document. It sorts the list of nodes you are processing; it does not reorder the source or the output tree anywhere else. To output sorted children inside a copy of the parent, apply templates to the sorted children inside that parent's template. If you use the identity pattern (XSLT basics: templates, match and apply), add a template for the parent that copies its attributes and applies templates to its children with an xsl:sort; the rest of the tree passes through the identity rule.

Numbers in the output

Sorting numerically and then printing 250000 or 44.00 shows the original text, because value-of of a node writes its string value. To format, use format-number(number(price), '#,##0.00'). In XSLT 1.0 the number type is a double and the pattern is the Java decimal-format one; locale-specific separators need an xsl:decimal-format declaration. Summing in the same stylesheet has the double-precision caveats from grouping records with XSLT 2 and Muenchian keys: if the figures are money, do the arithmetic where decimals exist.

Related reading

For the cluster overview see XSLT and XQuery for real transformations, and for passing the sort direction or field in from outside, passing parameters into an XSLT stylesheet. Sorting is specified in section 10 of XSLT 1.0, and the conversion of strings to numbers in the XPath 1.0 number function. The same operation in XQuery is order by (XQuery basics with BaseX). Test an XPath in the XPath tester.

When sorting should happen outside the stylesheet

The sort rule is complicated. Natural ordering ("item 2" before "item 10"), locale-aware ordering and multi-field business rules are easier and more testable in code.

You need the sorted data again. If another step consumes the sorted list, sort in the step that owns the data, not in a presentation stylesheet.

The list is large. A stylesheet sorts in memory after building the input tree. Sort in the database or stream the records.

The key needs computing. If the key is a calculation across several fields, compute it in the source system and sort on a stored column.