Reading Publisher Metadata in JATS XML for One Article

If you have a pile of journal articles as XML, the first job is usually not to read the paper but to get its metadata: title, authors, institutions, identifiers, dates. JATS (the Journal Article Tag Suite) is the vocabulary most publishers and repositories use, and its front matter is a rich, slightly awkward tree. This post reads one article's front matter, with the specific traps that make the first attempt return incomplete data, on a synthetic article.

The front matter

The parts you need, in the order they appear:

<article article-type="research-article">
 <front>
  <journal-meta>...<journal-title>Synthetic Letters</journal-title>...</journal-meta>
  <article-meta>
   <article-id pub-id-type="doi">10.9999/syn.0001</article-id>
   <title-group><article-title>A study of <italic>parsing</italic> in <sub>2</sub> steps</article-title></title-group>
   <contrib-group>
    <contrib contrib-type="author" corresp="yes">
      <contrib-id contrib-id-type="orcid">https://orcid.org/0000-0002-1825-0097</contrib-id>
      <name><surname>Doe</surname><given-names>Jane</given-names></name>
      <xref ref-type="aff" rid="aff1"/><xref ref-type="aff" rid="aff2"/></contrib>
    <contrib contrib-type="author"><name><surname>Roe</surname><given-names>Richard</given-names></name><xref ref-type="aff" rid="aff2"/></contrib>
    <contrib contrib-type="author"><collab>The Parsing Consortium</collab></contrib>
    <contrib contrib-type="editor"><name><surname>Poe</surname><given-names>Ed</given-names></name></contrib>
   </contrib-group>
   <aff id="aff1"><institution>University of Examples</institution>, <country>Germany</country></aff>
   <aff id="aff2"><institution>Institute of Samples</institution>, <country>France</country></aff>
   <pub-date date-type="pub" publication-format="electronic"><day>28</day><month>12</month><year>2024</year></pub-date>
   <pub-date date-type="pub" publication-format="print"><month>03</month><year>2025</year></pub-date>
   ...

Real JATS varies by version and by publisher: older files use pub-type instead of date-type, affiliations can sit inside the contrib, and some publishers add their own elements. The code below reads this form; check yours.

The title is mixed content

m = r.find("front/article-meta")
t = m.find("title-group/article-title")
print("title (itertext):", "".join(t.itertext()))
print("title (.text only):", repr(t.text))

title (itertext): A study of parsing in 2 steps
title (.text only): 'A study of '

The title has an italic word and a subscript inside it. .text returns only the text before the first child, 'A study of ', so a loop that stores .text truncates every title that contains markup. "".join(t.itertext()) returns the full plain text. If you want to keep the markup, serialize the element's children instead (mixed content in XML and why it hurts parsing). This single bug is the most common one in metadata extraction from JATS, and it is invisible: the titles look plausible, only shorter.

Authors, groups and editors

All contributors are contrib elements, and the role is an attribute:

affs = {a.get("id"): " ".join(a.xpath("string(institution)").split()) + ", " + a.findtext("country")
        for a in m.iterfind("aff")}
for c in m.iterfind("contrib-group/contrib"):
    role = c.get("contrib-type")
    nm = c.findtext("collab") or f"{c.findtext('name/given-names')} {c.findtext('name/surname')}"
    orcid = c.findtext("contrib-id[@contrib-id-type='orcid']")
    a = [affs[x.get("rid")] for x in c.iterfind("xref[@ref-type='aff']")]
    print(f"{role:7s} {nm:22s} orcid={orcid} affs={a}")

author  Jane Doe               orcid=https://orcid.org/0000-0002-1825-0097 affs=['University of Examples, Germany', 'Institute of Samples, France']
author  Richard Roe            orcid=None affs=['Institute of Samples, France']
author  The Parsing Consortium orcid=None affs=[]
editor  Ed Poe                 orcid=None affs=[]

Four points. First, the role: filtering on contrib-type="author" keeps three entries and drops the editor. Without the filter an author count is inflated. Second, a group author: the third author has a collab element and no name, so code that reads name/surname gets None and crashes or prints "None None". Handle the group case explicitly, and decide whether a consortium counts as one author or as many. Third, affiliations are indirect: the author holds an xref with a rid that matches an aff element's id. Building the dictionary once and looking up by rid gives the first author two institutions and the second one. Fourth, the ORCID appears only where supplied. None for the others is correct, and an analysis of "authors with an ORCID" is a count over the ones that have it. Keep the ORCID as the full URL or strip it to the 16-character identifier, but be consistent: the URL, https://orcid.org/0000-..., and the bare form both appear in the wild.

Names deserve caution. given-names and surname suit many cultures and not all: some names have no clear split, some are written family-name first. Store them as given, and do not build sort keys or initials from them without thinking about it.

Dates

An article has several dates, and they are different facts:

print("pub-dates:", [(d.get("publication-format"), d.findtext("year"), d.findtext("month")) for d in m.iterfind("pub-date")])
print("received:", m.xpath("string(history/date[@date-type='received']/year)"), "accepted:", m.xpath("string(history/date[@date-type='accepted']/year)"))

pub-dates: [('electronic', '2024', '12'), ('print', '2025', '03')]
received: 2024 accepted: 2024

The article was published online in December 2024 and in print in March 2025. Which is "the year" of the article? A count by year that uses the first pub-date it finds will mix the two across a corpus, depending on the order each publisher wrote them. Choose one, say which in the method, and select it by attribute. Dates may also be incomplete: the print date here has a month and a year, with no day. Do not pad to the first of the month and call it a date without noting it; store the parts you have. The history dates (received, accepted) give the editorial timeline, useful for studying review time.

Putting it into rows

For one article, three small tables: articles (DOI, title, journal, dates, volume, issue, pages), contributors (DOI, position, role, name or group, ORCID) and affiliations (DOI, contributor position, institution, country). Keep the contributor position, because author order carries meaning. For a collection, stream the files from the archive and append rows (analyzing publication metadata from XML archives), count the files that fail to parse, and count articles missing each field.

Tolerating variation across publishers

The code above reads one shape. In a corpus, expect each publisher to differ, and build the reader so the differences are visible and not fatal. Wrap each field in a small function that tries the known forms in order and returns None if none matches: the title from title-group/article-title, falling back to article-title under book-part for book chapters; the publication year from the electronic date, then the print date, then any pub-date. Log which fallback fired, and count them by source. The counts tell you where your assumptions are weakest.

Test with a sample from each source before processing the whole archive. Take twenty files per publisher, run the reader, and print every field for a few. Look for empty titles, authors without names, years outside a plausible range, DOIs with whitespace or a doi: prefix and affiliations that are empty strings. Each of these is a rule to add, and each is cheaper to find on twenty files than on two million.

Keep the version of the vocabulary in the output. JATS 1.0, 1.1, 1.2 and 1.3 differ in small ways, and the dtd-version attribute on the article element says which one a file uses. Recording it lets you explain a difference between two batches without opening the files, and lets you target a rule at the versions it applies to.

Related reading

This post belongs to XML problems worked from real data. The XPath behind the predicates is in XPath predicates and position. The standard is documented at the JATS website with the element reference in the JATS tag library, and identifiers at ORCID. Check a path in the XPath tester.

When a JATS front matter reader is not enough

You need the full text. Body sections, figures, tables and references are a second job with a harder structure. Start from the tag library, and keep the inline markup.

You need clean author identities. Matching authors across papers is a disambiguation problem. ORCID helps where present and is often absent.

Your files come from many publishers. Each uses JATS a little differently, so test on a sample from each, and keep per-source differences in configuration.

The records are in a different vocabulary. Biomedical records may be in the PubMed XML format, and library records in MARCXML or Dublin Core, with their own field names and rules.