Debian Codegraph @

texmf-dist/doc/latex/tagpdf/tagpdf.tex

raw
1 % !Mode:: "TeX:DE:UTF-8:Main"
2 \PassOptionsToPackage{check-declarations,enable-debug}{expl3}
3
4 % Note on the compilation of the documentation:
5 % The documentation uses for the tagging sometimes code
6 % that is under development and/or not public yet.
7 % To compile an *untagged* documentation, replace tagging=on by
8 % tagging=of in the following \DocumentMetadata command.
9
10 \DocumentMetadata
11 {
12 % change the following line to tagging=off to compile an untagged documentation
13 tagging=on,
14 tagging-setup={debug/parent-child-check=atend,tabsorder=structure},
15 pdfversion=2.0,lang=en,pdfstandard=a-4f,pdfstandard=ua-2,
16 %uncompress
17 }
18 \DebugBlocksOff
19 \makeatletter
20 \def\UlrikeFischer@package@version{1.0c}
21 \def\UlrikeFischer@package@date{2026-05-17}
22 \makeatother
23
24 \documentclass[bibliography=totoc,a4paper]{article}
25
26 \usepackage{geometry}
27 \usepackage[english]{babel}
28
29 \usepackage{unicode-math}
30 \setmainfont{Heuristica}
31 \usepackage[nopatch]{microtype}
32
33
34 \usepackage[autostyle]{csquotes}
35 \usepackage[style=numeric]{biblatex}
36 \addbibresource{tagpdf.bib}
37
38 \reversemarginpar
39 \NewDocumentCommand\sidenote{m}{\marginpar{#1}}
40 \usepackage{booktabs}
41 \setlength\belowcaptionskip{10pt}
42 \usepackage{tcolorbox}
43
44 \usepackage{tikz}
45 \usetikzlibrary{positioning}
46 \usetikzlibrary{fit,tikzmark}
47 \usetikzlibrary{arrows.meta}
48 \tikzset{arg/.style = {font=\footnotesize\ttfamily, anchor=base,draw, rounded corners,node distance=2mm and 2mm}}
49 \tikzset{operator/.style = {font=\footnotesize\ttfamily, anchor=base,draw, rounded corners,node distance=4mm and 4mm}}
50
51
52 \usepackage{listings}
53 \lstset{basicstyle=\ttfamily, columns=fullflexible,language=[LaTeX]TeX,
54 escapechar=*,
55 commentstyle=\color{green!50!black}\bfseries}
56
57 % this allow to get real spaces in the code parts.
58 % This should perhaps be combined in a new listings key
59 \lstset{showspaces}
60 \makeatletter \def\lst@visiblespace{\lst@ttfamily{\char32}{\char32}}\makeatother
61
62
63 \usepackage{hyperref}
64
65 \hypersetup{
66 pdftitle={The tagpdf package, v\csname UlrikeFischer@package@version\endcsname},
67 pdfauthor=Ulrike Fischer,
68 colorlinks}
69
70 \tcbuselibrary{documentation}
71 \definecolor{Definition}{rgb}{0,0.2,0.6}
72 \newcommand\PrintKeyName[1]{\textsf{#1}}
73 \newcommand\pkg[1]{\texttt{#1}}
74 \newcommand\DescribeKey[1]{\texttt{#1}}
75
76 %tagging patches:
77 \usepackage{longtable}
78 \usepackage{tagpdfdocu-patches}
79
80 \newcommand\PDF{PDF}
81
82 \title{The \pkg{tagpdf} package, v\csname UlrikeFischer@package@version\endcsname}
83 \date{\csname UlrikeFischer@package@date\endcsname}
84 \author{Ulrike Fischer\thanks{[email protected]}}
85
86 \usepackage{shortvrb}
87 \MakeShortVerb|
88
89 \begin{document}
90
91 \maketitle
92
93 \begin{tcolorbox}[colframe=red]
94
95 This package is not meant for direct use in document preambles. It is loaded automatically
96 if \cs{DocumentMetadata} is used.
97
98 The package started in 2018 as a tool to \emph{research} tagging. It is now the base of the code developed
99 in the \pkg{latex-lab} bundle for the Tagged PDF project (i.e., loaded by that code)
100 \url{https://latex3.github.io/tagging-project/}.
101
102 The package is developed and improved in parallel with the \LaTeX{} kernel, the code in
103 the \pkg{latex-lab} bundle (part of the core \LaTeX{} distribution), the \pkg{pdfmanagement-testphase}
104 package (the \LaTeX{} PDF management bundle) and the L3 programming layer (part of the \LaTeX{} format).
105 That means you must ensure that all these components are up-to-date and in
106 sync with each other.
107
108 This package quite probably still contains some bugs. It is in some parts quite slow because
109 the code currently prefers readability over speed. At some point in the future its code will
110 be integrated into the \LaTeX{} format and then this package will disappear.
111
112 Because of its function as a research and development tool it is
113 important to understand that this package can still change in
114 incompatible ways from one version to the next.
115
116 You need some knowledge about \TeX, \PDF{} and perhaps even Lua to use its commands
117 directly.
118
119 \medskip
120
121 Issues, comments, suggestions can be added as issues to these two github tracker:
122
123 \medskip
124 \centering \url{https://github.com/latex3/tagging-project}\par
125 \leavevmode\llap{or\qquad\qquad} \url{https://github.com/latex3/tagpdf}
126
127 \end{tcolorbox}
128
129
130
131 \bookmark[dest=toc,level=section]{Table of Contents}
132 \NextLinkTarget{toc}
133 \tableofcontents
134
135
136
137 \section{Introduction}
138
139 For many years the creation of accessible, tagged \PDF{}-files with \LaTeX\
140 that conform to the PDF/UA standard has been on the agenda of \TeX-meetings.
141 Many people agree that this is important and Ross Moore has done quite some
142 work on it. There is also a TUG-mailing list and a web page
143 \parencite{tugaccess} dedicated to this topic.
144
145 In my opinion missing were means to \emph{experiment} with tagging and
146 accessibility. Means to try out, how difficult it is to tag some structures,
147 means to try out, how much tagging is really needed (standards and validators
148 don't need to be right \ldots), means to test what else is needed so that a
149 \PDF{} works e.g. with a screen reader, means to try out how core \LaTeX\
150 commands behave if tagging is used. Without such experiments it is in my
151 opinion quite difficult to get a feeling about what has to be done, which
152 kernel changes are needed, and how packages should be adapted.
153
154 This package was developed to close this gap by offering \emph{core} commands
155 to tag a \PDF{}\footnote{In case you don't know what this means: there will
156 be some explanations later on.}. My hope was that the knowledge gained by the
157 use of this package would in the end allow to decide if and how code to do
158 tagging should become part of the \LaTeX\ kernel.
159
160
161 The code has been written so that it can be added as a module to the \LaTeX{}
162 kernel itself if it turned out to be usable. It therefore avoids patching
163 commands from other packages. It was also not an aim of the package to
164 develop patches to directly enable tagging in other packages. While in the end changes to various commands in many
165 classes and packages will be needed to automatically get tagged \PDF{} files, these changes
166 should be done by class, package and document writers themselves using a
167 sensible API provided by the kernel and not by some external package that
168 adds patches everywhere and would need constant maintenance --- one only needs
169 to look at packages like \pkg{tex4ht} or \pkg{bidi} or \pkg{hyperref} to see how difficult and
170 sometimes fragile this is.
171
172 The package is now a part of the Tagged PDF project and triggered already
173 various changes in the \LaTeX\ kernel and the engines: There is a new PDF
174 management, the new para hooks allow to automatically tag paragraphs, after
175 changes in the output routine page breaks and header and footer are handled
176 correctly, the engines now support structure destinations. More changes are
177 in the latex-lab bundle and are loaded through the \texttt{tagging} keys.
178
179 I'm sure that tagpdf still has bugs. Bug reports, suggestions and comments
180 can be added to the issue tracker on github either
181 \url{https://github.com/latex3/tagpdf} or
182 \url{https://github.com/latex3/tagging-project}.
183
184 Please also check the github site and latex-lab for new examples and improvements.
185
186 \subsection{Tagging and accessibility}
187
188 While the package is named \pkg{tagpdf} the goal is also \emph{accessible}
189 \PDF{}-files. Tagging is \emph{one} (the most difficult) requirement for
190 accessibility but there are others. I will mention some later on in this
191 documentation, and -- if sensible -- also add code, keys or
192 tips for them.
193
194 So the name of the package is a bit wrong. As excuse I can only say that it
195 is short and easy to pronounce (and of course, it was always meant to be temporary).
196
197
198 \subsection{Engines and modes}
199
200 Theoretically, the package works with all engines, but the xelatex and the
201 latex-dvips-route are basically untested and they also don't support real
202 space glyphs so I don't recommend them. lualatex is the most powerful and
203 safe modus and should be used for new documents, it is slower than pdflatex
204 but requires less compilations. pdflatex works ok and can be used for legacy
205 documents; it needs more compilations to resolve all cross references needed
206 for the tagging.
207
208 The package has two modes: the \emph{generic mode} which should work in
209 theory with every engine and the \emph{Lua mode} which works only with
210 lualatex and (since version 0.98k) with dvilualatex. Since version 0.99m,
211 the Lua mode is forced if luatex is detected, otherwise generic mode is used.
212
213 I implemented the generic mode first. Mostly because my \TeX\ skills are much
214 better than my Lua skills and I wanted to get the \TeX\ side right before
215 starting to fight with attributes and node traversing.
216
217 While the generic mode is not bad and I spent quite some time to get it
218 working I nevertheless think that the Lua mode is the future and the only one
219 that will be usable for larger documents. \PDF{} is a page orientated format
220 and so the ability of luatex to manipulate pages and nodes after the
221 \TeX-processing has finished is really useful here. Also with luatex characters are
222 normally already given as Unicode.
223
224 The package uses quite a lot labels (in generic mode more than with luamode).
225 It is now based on the property module of the \LaTeX{} kernel. This module
226 provides expandable references but the drawback is that (right now) they don't always give
227 good rerun messages if they have changed. I advise to use the
228 \pkg{rerunfilecheck} package as an intermediate work-around and when using
229 pdflatex compile at least once or twice more often than normal.
230
231
232 \subsection{References and target PDF version}
233
234 My main reference for the first versions of this package was the free
235 reference for \PDF{} 1.7. \parencite{pdfreference} and so they implemented
236 only support for \PDF{} 1.7.
237
238 In 2018 \PDF{} 2.0. has been released. The reference can now be bought at no
239 cost through the PDF association.
240
241 \PDF{} 2.0 has a number of features that are really needed for good tagging:
242 it knows more structure types, it allows to add associated files to
243 structures---these are small, embedded files that can, for example, contain
244 the mathML or source code of an equation---, it knows structure destinations,
245 which allows to link to a structure. It knows the MathML namespace.
246
247 \LaTeX{} therefore targets \PDF{} 2.0 and tagpdf has support for
248 associated files, for name spaces and other \PDF{} 2.0 features.
249
250 \PDF{}~2.0 features are currently (begin of 2025) still not well supported by
251 \PDF~consumer, but some progress has been made. Foxit can handle MathML associated files
252 and to some extend MathML structure elements and
253 together with development versions of NVDA and MathCat reading of equations is already quite good. The PDF Accessibility Checker (PAC) no longer crashes
254 if one tries to load a \PDF{} 2.0 file. We recommend
255 to use \PDF{} 2.0 if possible and then to complain to the PDF{} consumer if
256 something doesn't work.
257
258 The package doesn't try to suppress all 2.0 features if an older \PDF{}
259 version is produced. It normally doesn't harm if a \PDF{} contains keys
260 unknown in its version and it makes the code faster and easier to maintain if
261 there aren't too many tests and code paths; so for example associated files
262 will always be added. But tests could be added in case this leads to
263 incompatibilities.
264
265
266 \subsection{Validation}
267
268 \PDF{}'s created with the commands of this package must be validated:
269
270 \begin{itemize}
271 \item
272 One must check that the \PDF{} is \emph{syntactically} correct.
273 It is rather easy to create broken \PDF{}:
274 e.g. if a chunk is opened on one page but closed
275 on the next page or if the document isn't compiled often enough.
276
277 \item One must check how good the PDF follows requirements of standards
278 like PDF/UA \emph{formally}\footnote{The PDF/UA-2 standard for \PDF~2.0
279 will hopefully be released begin of 2024.}.
280
281 \item
282 One must check how good the accessibility is \emph{practically}.
283 \end{itemize}
284
285
286 Syntax validation and formal standard validation can be done with the
287 validator veraPDF \parencite{verapdf} which can also handle PDF 2.0 files.
288 Other options (only for PDF 1.7 and older) are preflight of the (non-free) Adobe Acrobat and the free \PDF{} Accessibility Checker (PAC~2024) \parencite{pac2024}.
289 A quite useful tool
290 is \enquote{Next Generation PDF} \parencite{ngpdf}, a browser application
291 which converts a tagged PDF to html, allows to inspect its structure and also
292 to edit the structure. For PDF~2.0 files there is also a checker based on the
293 Arlington model from veraPDF.
294
295 A tool developed by the \LaTeX{} team allows to extract the structure as XML and to validate it against a schema. This can be tested as \url{https://texlive.net/showtags}.
296
297 Practical validation is naturally the more complicated part.
298 It needs screen reader, users which actually knows how to handle them,
299 can test documents and can report where a \PDF{} has real accessibility problems.
300
301
302 \minisec{Preflight woes}
303
304 Sadly validators can not be always trusted.
305 As an example for an reason that I don't understand the adobe preflight
306 don't like the list structure \texttt{L}.
307 It is also possible that validators contradict: that the one says everything is okay,
308 while the other complains. Generally when in doubt I recommend to use and trust verapdf.
309
310 \subsection{Examples wanted!}
311
312 To make the package usable examples are needed: examples that demonstrate how
313 various structures can be tagged and which patches are needed, examples for
314 the test suite, examples that demonstrates problems.
315
316
317 \begin{tcolorbox}
318 Feedback, contributions and corrections are welcome!
319 \end{tcolorbox}
320
321
322 All examples should use the \cs{DocumentMetadata} key \PrintKeyName{uncompress}
323 so that uncompressed \PDF{} are created and the internal objects and
324 structures can be inspected and be compared by the l3build checks.%
325
326
327
328 \subsection{Proof of concept: the tagging of the documentation itself}
329
330 Starting with version 0.6 the documentation itself has been tagged. The
331 tagging wasn't (and isn't) in no way perfect. The validator from Adobe didn't
332 complain, but PAC~3 wanted alternative text for all links (no idea why) and
333 so I put everywhere simple text like \enquote{link} and \enquote{ref}. The
334 links to footnotes gave warnings, so I disabled them. I used types from the
335 \PDF{} version 1.7, mostly as I had no idea what should be used for code in
336 2.0. Margin notes were simply wrong and there were tagging commands
337 everywhere \ldots
338
339 The tagging has been improved and automated over time in sync with
340 improvements and new features in the \LaTeX\ kernel, the latex-lab bundle and
341 the \PDF\ management code and is now much better. Only a few
342 structures---mostly some from currently unsupported packages--- still need
343 manual tagging. But sadly the output of the validators don't quite reflect
344 the improvements. The documentation uses now \PDF~2.0 and while the newest
345 PAC~2024 can at least open the file it can not validate properly the file.
346 It doesn't fully supports \PDF~2.0 (see figure~\ref{fig:pac-version}). It also
347 complains about the tabular header cells as it doesn't follow attribute classes, see
348 figure~\ref{fig:pac-validation}.
349 The Adobe validator has a bug and
350 doesn't like the (valid) use of the \texttt{Lbl} tag for numbers outside lists and
351 so complains about the tagging of the table of content entries (see figure~\ref{fig:adobe}).
352
353
354 But even if the documentation would pass all the tests of the validators: as
355 mentioned above passing a formal test doesn't mean that the content is really
356 good and usable. The user commands used for the tagging and also some of the
357 patches used are still rather crude. So there is lot space for improvement.
358
359 \begin{tcolorbox}[]
360 Be aware that to create the tagged version a current lualatex-dev and a
361 current version of the pdfmanagment-testphase package is needed.
362 \end{tcolorbox}
363
364 \begin{figure}
365 \includegraphics[alt=Screenshot showing that PAC 2024 doesn't support PDF 2.0]{pac2024-version}
366 \caption{PAC 2024 reading a PDF 2.0 file}\label{fig:pac-version}
367 \end{figure}
368
369 \begin{figure}
370 \includegraphics[alt=PAC 2024 complains about table header cells]{pac2024-report}
371 \caption{PAC 2024 Validation of header cells}\label{fig:pac-validation}
372 \end{figure}
373
374 \begin{figure}
375 \includegraphics[alt={Screenshot of Adobe report}]{acrobat}
376 \caption{Adobe Acrobat complaining
377 about the \texttt{Lbl} use}\label{fig:adobe}
378 \end{figure}
379
380
381 \section{Loading}
382
383 The package requires the new PDF management. With a current \LaTeX{} (2025-06-01 or newer)
384 the PDF management is loaded if you use the \cs{DocumentMetadata} command before \cs{documentclass}.
385 The \pkg{tagpdf} package can then be loaded and activated by using the \texttt{tagging=on} key. The exact behavior of
386 the \texttt{tagging} key is documented in \texttt{documentmetadata-support-doc.pdf} which
387 is part of the \pkg{latex-lab} bundle.
388
389 Various parts of the code differentiate between \PDF{} version 2.0 and lower versions. By default \cs{DocumentMetadata} will now set the \PDF{} version to 2.0.
390 If \PDF{} 1.7 is wanted it is required to set the version early in the first \cs{DocumentMetadata}
391 command so that \pkg{tagpdf} can pick up the correct code path.
392
393 \begin{lstlisting}
394 \DocumentMetadata
395 {
396 tagging=on,
397 pdfstandard=ua-2, % pdfstandard can be set too
398 }
399 \documentclass{article}
400 \begin{document}
401 some text
402 \end{document}
403 \end{lstlisting}
404
405 \minisec{Deactivation}
406
407 When loading \pkg{tagpdf} through the \texttt{tagging=on} keys, it is automatically activated.
408 To deactivate it while still retaining all the other new code from the latex-lab files, you can use the \texttt{tagging=off} key.
409
410 Starting with \LaTeX{} 2025-11-01, you can also use |\DocumentMetadata{}|, which will
411 then do the same as |\DocumentMetadata{tagging=off}|. This also deactivates the interword space code.
412
413 \minisec{Loading as package needs activation!}
414
415 It is not recommended anymore, but the package can also be loaded
416 normally with |\usepackage| or |\RequirePackage|. But it still requires
417 the \PDF\ management, so you need to do something like this:
418 \begin{lstlisting}
419 \RequirePackage{pdfmanagement}
420 \RequirePackage{tagpdf}
421 \end{lstlisting}
422
423 But it will then -- apart from loading more packages and defining a lot of things
424 -- not do much. You will have to \emph{activate} it
425 with \verb+\tagpdfsetup+.
426
427 The \PDF\ management will in any
428 case load \pkg{tagpdf-base} a small package that provides no-op
429 versions of the main tagging commands.
430
431 Most commands do nothing if tagging is not activated, but in case a
432 test is needed a command (with the usual p,T,F variants) is provided:
433
434 \begin{docCommand}{tag_if_active:TF}{}\end{docCommand}
435
436 The check is true only if \emph{everything} is activated. In all other
437 cases (including if tagging has been stopped locally) it will be
438 false.
439
440 \subsection{Modes and package options}
441
442
443 %TODO think about tagging of the keys. Aside? Header?
444
445
446 The package has two different modes: The \textbf{generic mode} works
447 (in theory, currently only fully tested with pdflatex) probably with
448 all engines, the \textbf{Lua mode} only with lualatex. The differences
449 between both modes will be described later.
450
451 Starting with version 0.99m the mode is set automatically (Lua mode for luatex, generic mode otherwise). The package options do nothing anymore and will be removed in future versions.
452
453
454 \subsection{Setup and activation}\label{ssec:setup}
455
456
457 \begin{docCommand}{tagpdfsetup}{\marg{key-val-list}}\end{docCommand}
458
459 This command setups the general behavior of the package.
460 The command should be normally used only in the preamble
461 (for a few keys it could also make sense to change them in the document).
462
463 The key-val list understands at least the following keys. More keys are defined in some of the latex-lab module, see table~\ref{tab:setupkey} for an overview which also includes older, now deprecated names.
464
465 The command is deactivated before the class is loaded if the key |tagging=off| is used in \cs{DocumentMetadata} and then does nothing.
466
467 \begin{description}
468 \item[\PrintKeyName{activate/all}] Boolean, initially false. Activates
469 everything, that's normally the sensible thing to do.
470
471 \item [\PrintKeyName{activate}] Like |activate/all|,
472 \emph{additionally} is opens at begin document a structure with
473 |\tagstructbegin| and closes it at end document. The key accepts as
474 value a tag name which is used as the tag of the structure. The
475 default value is |Document|.
476
477 \item[\PrintKeyName{activate/mc}] Boolean, initially false. Activates
478 the code related to marked content.
479
480 \item[\PrintKeyName{activate/struct}] Boolean, initially
481 false. Activates the code related to structures. Should be used only
482 if \PrintKeyName{activate/mc} has been used too.
483
484 \item[\PrintKeyName{activate/struct-dest}] Boolean, initially true.
485 Starting with version 0.93
486 \pkg{tagpdf} will create automatically structure destinations (see
487 section~\ref{sec:struct-dest} if \pkg{hyperref} is used and if the
488 engine supports it. With this key this
489 can be suppressed.
490
491 \item[\PrintKeyName{activate/tree}] Boolean, initially
492 false. Activates the code related to trees. Should be used only if
493 the two other keys has been used too.
494
495 \item[\PrintKeyName{activate/spaces}] Boolean. The key
496 activates/deactivates the insertion of space glyphs, see
497 section~\ref{sec:spacechars}. In the luamode it only works if at
498 least \PrintKeyName{activate/mc} has been used. It is on by default if you use
499 |tagging=on| in \cs{DocumentMetadata}. The |tagging=off| key of \cs{DocumentMetadata} deactivates it.
500 The old name of the key |interwordspace| is still supported but deprecated.
501
502 \item[\PrintKeyName{activate/softhyphen}] luamode only.
503 The key activates and controls the replacing of hard hyphens from hyphenation
504 by soft hyphens. It knows the values
505 \begin{itemize}
506 \item \texttt{false} and \texttt{off} deactivates the automatic handling.
507 \item \texttt{char} Replaces the hyphens by U+00AD if the font contains this glyph.
508 Note that in some newer fonts U+00AD is an invisible character and so this could led to disappearing hyphens.
509 \item \texttt{artifact} This keeps the hard hyphen U+002D but
510 surrounds it by an Artifact-MC if tagging. This only works if the document is tagged. It requires that the \PrintKeyName{active/tagunmarked} key is not set to false.
511 \item \texttt{true}, \texttt{on} do currently the same as \texttt{artifact}.
512 \end{itemize}
513
514 If tagging is not activate the softhyphen handling is not activated, by default and the hyphen stays a hard-hyphen. If is possible to use the \texttt{char} mode with
515 \begin{verbatim}
516 \DocumentMetadata{tagging=off,tagging-setup={activate/softhyphen=char},}
517 \end{verbatim}
518 As \cs{tagpdfsetup} is deactivated in the document is not possible to
519 change the mode locally in the document with it.
520
521 If tagging is active the default is the \texttt{artifact} mode. The mode can be changed with \cs{tagpdfsetup} at any time. Switching on and off the softhyphen is a global operation, a switch between artifact and char more is a local operation.
522
523 \item[\PrintKeyName{role/new-tag}] Allows to define new tag names, see
524 section \ref{sec:new-tag} for a description.
525
526 \item[\PrintKeyName{role/new-attribute}] This key takes two arguments and
527 declares an attribute. See \ref{sec:attributes}.
528
529 \item[\PrintKeyName{role/map-tags}] This key allows to remap the structure
530 tags. Currently it supports only two values: |false| (the default) and |pdf| which
531 maps all tags to their standard PDF role, e.g. |itemize| will be mapped to |L|.
532
533
534 \item[\PrintKeyName{para/tagging}] Boolean. This activate/deactivates
535 the automatic tagging of paragraphs, see \ref{sec:paratagging} for
536 more background. It uses the \texttt{para/begin} and
537 \texttt{para/end} hooks.
538 With more tagging support conditions will be added, that means the
539 code is bound to change! Paragraphs can appear in many unexpected
540 places and the code can easily break, so there is also an option to
541 see where such paragraphs are:
542
543
544 \item[\PrintKeyName{para/tag}] String. This key changes the second tag
545 used by the paratagging code. The default tag is \texttt{text}, a
546 \LaTeX{} specific tag that is role mapped to \texttt{P}. A useful
547 local setting here can be \texttt{NonStruct}, which creates a
548 structure \enquote{without meaning}.
549
550 \item[\PrintKeyName{para/maintag}] String. This key changes the first tag
551 used by the paratagging code. The default tag is \texttt{text-unit}, a
552 \LaTeX{} specific tag that is role mapped to \texttt{Part}.
553
554 \item[\PrintKeyName{page/tabsorder}] Choice key, possible values are
555 \PrintKeyName{row}, \PrintKeyName{column}, \PrintKeyName{structure},
556 \PrintKeyName{none}. This decides if a \verb+/Tabs+ value is
557 written to the dictionary of the page objects. Not really needed for
558 tagging itself, but one of the things you probably need for
559 accessibility checks. So I added it. Currently the tabsorder is the
560 same for all pages. Perhaps this should be changed \ldots.
561
562 \item[\PrintKeyName{activate/tagunmarked}] Boolean,\sidenote{luamode} initially
563 true. When this boolean is true, the Lua code will try to mark
564 everything that has not been marked yet as an artifact. The benefit
565 is that one doesn't have to mark up every deco rule oneself. The
566 danger is that it perhaps marks things that shouldn't be marked --
567 it hasn't been tested yet with complicated documents containing
568 annotations etc.
569
570 \item[\PrintKeyName{viewer/startstructure}] A structure number. If a \texttt{OpenAction}
571 is set in the PDF Catalog (which is normally the case if hyperref is used)
572 a structure destination pointing to the structure is added. The initial value is structure 1 (the \texttt{Document} structure), the default value is the current structure. The
573 key can be used more than once, the last setting will win.
574
575
576 \item[\PrintKeyName{debug/uncompress}] Sets both the \PDF{} compresslevel
577 and the \PDF{} objcompresslevel to 0 and so allows to inspect the
578 \PDF{}. No really useful anymore as this can also
579 be set in \cs{DocumentMetadata}.
580
581 \item[\PrintKeyName{debug}] This keys knows a number of sub-keys to
582 set various debug options.
583 \begin{description}
584 \item[\PrintKeyName{debug/show}] This takes a comma list of keywords:
585
586 \texttt{spaces}/\texttt{spacesOff}: \sidenote{luamode}
587 That helps in Lua mode to see where space glyph will be
588 inserted if \PrintKeyName{activate/spaces} is activated.
589 This can also be activated with the now deprecated key |show-spaces|
590
591 \texttt{para}/\texttt{paraOff}: This (locally)
592 activates/deactivates small red and green numbers in the places where
593 the paratagging hook code is used.
594
595
596 \item[\PrintKeyName{debug/log}] Choice key, possible values
597 \PrintKeyName{none}, \PrintKeyName{v}, \PrintKeyName{vv},
598 \PrintKeyName{vvv}, \PrintKeyName{all}. Setups the log level.
599 Changing the value affects currently mostly the luamode:
600 \enquote{higher} values gives more messages in the log. The current
601 levels and messages have been setup in a quite ad-hoc manner and
602 will need improvement.
603
604
605 \end{description}
606
607 \end{description}
608
609 \input{tagpdfsetup-keys}
610
611 \begin{docCommands}
612 {
613 {doc name=tagtool,doc parameter=\marg{key-val}},
614 {doc name=tag_tool:n,doc parameter=\marg{key-val}}
615 }
616 \end{docCommands}
617
618 These commands are deprecated. Tagging sockets (documented in source2e) should be used instead.
619
620
621 \section{Tagging}
622
623
624 PDF is a page orientated graphic format. It simply puts ink and glyphs
625 at various coordinates on a page. A simple stream of a page can look
626 like this\footnote{The appendix contains some remarks about the syntax
627 of a \PDF{} file}:
628
629 \begin{lstlisting}[columns=fixed]
630 stream
631 BT
632 /F27 14.3462 Tf %select font
633 89.291 746.742 Td %move point
634 [(1)-574(Intro)-32(duction)]TJ %print text
635 /F24 10.9091 Tf %select font
636 0 -24.35 Td %move point
637 [(Let's)-331(start)]TJ %print text
638 205.635 -605.688 Td %move point
639 [(1)]TJ %print text
640 ET
641 endstream
642 \end{lstlisting}
643
644
645 From this stream one can extract the characters and their placement on the page
646 but not their semantic meaning (the first line is actually a section heading,
647 the last the page number). And while in the example the order is correct
648 there is actually no guaranty that the stream contains the text in the order
649 it should be read.
650
651 Tagging means to enrich the \PDF{} with information about the \emph{semantic}
652 meaning and the \emph{reading order}. (Tagging can do more, one can also
653 store all sorts of layout information like font properties and indentation
654 with tags. But as I already wrote this package concentrates on the part of
655 tagging that is needed to improve accessibility.)
656
657
658 \subsection{Three tasks}
659
660
661 To tag a \PDF{} three tasks must be carried out:
662
663 \begin{enumerate}
664 \item
665 \textbf{The mark-content-task}:\sidenote{mc-task} The document must add
666 \enquote{labels} to the page stream which allows to identify and reference
667 the various chunks of text and other content.
668 This is the most difficult part of tagging -- both for the document writer
669 but also for the package code. At first there can be quite many
670 chunks as every one is a leaf node of the structure and so often a rather
671 small unit. At second the chunks must be defined page-wise -- and
672 this is not easy when you don't know where the page breaks are.
673 Also in a standard document a lot text is created automatically, e.g.
674 the toc, references, citations, list numbers etc and it is not always
675 easy to mark them correctly.
676
677 \item \textbf{The structure-task}:\sidenote{struct-task} The document must
678 declare the structure. This means marking the start and end of
679 semantically connected portions of the document (correctly nested as a
680 tree). This too means some work for the document writer, but less than
681 for the mc-task: at first quite often the mc-task and the
682 structure-task can be combined, e.g. when you mark up a list number or
683 a tabular cell or a section header; at second one doesn't have to worry
684 about page breaks so quite often one can patch standard environments to
685 declare the structure. On the other side a number of structures end in
686 \LaTeX\ only implicitly -- e.g. an item ends at the next item, so
687 getting the \PDF{} structure right still means that additional mark up
688 must be added.
689
690
691 \item \textbf{The tree management}:\sidenote{tree-task} At last the
692 structure must be written into the \PDF{}. For every structure an
693 object of type \texttt{StructElem} must be created and flushed with
694 keys for the parents and the kids. A parent tree must be created to get
695 a reference from the mc-chunks to the parent structure. A role map must
696 be written. And a number of dictionary entries. All this is hopefully
697 done automatically and correctly by the package \ldots.
698
699 \end{enumerate}
700
701
702 \begin{figure}[t!]
703
704
705 \begin{tcolorbox}[]
706
707 \minisec{Page stream with marked content}
708
709 \begin{tikzpicture}[baseline=(a.north),node distance=2pt,remember picture,
710 alt={Illustration of page stream with marked content}]
711 \node(start){\ldots~\ldots~\ldots};
712 \node[draw,base right = of start](a) {mc-chunk 1};
713 \node[draw,base right = of a](b) {mc-chunk 2};
714 \node[draw,base right = of b](c) {mc-chunk 3};
715 \node[draw,base right = of c](d) {mc-chunk 3};
716 \node[base right = of d] {\ldots~\ldots};
717 \end{tikzpicture}
718
719
720 \minisec{Structure}
721
722
723 \newlength\ydistance\setlength\ydistance{-0.8cm}
724 \begin{tikzpicture}[remember picture,baseline=(root.north),alt={Illustration of structure}]
725 \node[draw,anchor=base west] (root) at (0,0) {Sect (start section)};
726 \node[draw,anchor=base west] at (0.3,\ydistance) {H (header section)};
727 \node[draw,anchor=base west](aref) at (0.6,2\ydistance){mc-chunk 1};
728 \node[draw,anchor=base west](bref) at (0.6,3\ydistance){mc-chunk 2};
729 \node[draw,anchor=base west] at (0.3,4\ydistance){/H (end header)};
730 \node[draw,anchor=base west] at (0.3,5\ydistance){P (start paragraph)};
731 \node[draw,anchor=base west](cref) at (0.6,6\ydistance){mc-chunk 3};
732 \node[draw,anchor=base west](dref) at (0.6,7\ydistance){mc-chunk 4};
733 \node[draw,anchor=base west] at (0.3,8\ydistance){/P (end paragraph)};
734 \node[draw,anchor=base west] at (0,9\ydistance){/Sect (end section)};
735 \end{tikzpicture}
736 \begin{tikzpicture}[remember picture, overlay]
737 \draw[->,red](aref)-|(a);
738 \draw[->,red](bref)-|(b);
739 \draw[->,red](cref)-|(c);
740 \draw[->,red](dref)-|(d);
741 \end{tikzpicture}
742
743 \end{tcolorbox}
744
745 \caption{Schematical
746 description of the relation between marked content in the page stream and the
747 structure}
748
749 \end{figure}
750
751
752
753
754 \subsection{Task 1: Marking the chunks: the mark-content-step}
755
756
757
758 To be able to refer to parts of the text in the structure, the text in the
759 page stream must get \enquote{labels}. In the \PDF{} reference they are
760 called \enquote{marked content}. The three main variants needed here are:
761
762 \begin{description}
763 \item[Artifacts] They are marked with of a pair of keywords, \texttt{BMC}
764 and \texttt{EMC} which surrounds the text. \texttt{BMC} has a single
765 prefix argument, the fix tag name \texttt{/Artifact}. Artifacts should
766 be used for irrelevant text and page content that should be ignored in
767 the structure. Sadly it is often not possible to leave such text simply
768 unmarked -- the accessibility tests in Acrobat and other validators
769 complain.
770
771
772 \begin{lstlisting}
773 /Artifact BMC
774 text to be marked
775 /EMC
776 \end{lstlisting}
777
778
779
780
781
782 \item[Artifacts with a type] They are marked with of a pair of keywords,
783 \texttt{BDC} and \texttt{EMC} which surrounds the text. \texttt{BDC}
784 has two arguments: again the tag name \texttt{/Artifact} and a
785 following dictionary which allows to specify the suppressed info. Text
786 in header and footer can e.g. be declared as pagination like this:
787
788
789 \begin{lstlisting}
790 /Artifact <</Type /Pagination>> BDC
791 text to be marked
792 /EMC
793 \end{lstlisting}
794
795
796
797
798 \item[Content] Content is marked also with of a pair of keywords,
799 \texttt{BDC} and \texttt{EMC}. The first argument of \texttt{BDC} is a
800 tag name which describes the structural type of the text\footnote{There
801 is quite some redundancy in the specification here. The structural type
802 is also set in the structure tree. One wonders if it isn't enough to
803 use always \texttt{/SPAN} here.} Examples are \texttt{/P} (paragraph),
804 \texttt{/H2} (heading), \texttt{/TD} (table cell). The reference
805 mentions a number of standard types but it is possible to add more or
806 to use different names.
807
808 In the second argument of \texttt{BDC} -- in the property dictionary -- more
809 data can be stored. \emph{Required} is an \texttt{/MCID}-key which takes an
810 integer as a value:
811
812
813 \begin{lstlisting}
814 /H1 <</MCID 3>> BDC
815 text to be marked
816 /EMC
817 \end{lstlisting}
818
819
820 This integer is used to identify the chunk when building the structure
821 tree. The chunks are numbered by page starting with 0. As the numbers are
822 also used as an index in an array they shouldn't be \enquote{holes} in the
823 numbering system (It is perhaps possible to handle a numbering scheme not
824 starting by 0 and having holes, but it will enlarge the \PDF{} as one would
825 need dummy objects.).
826
827 It is possible to add more entries to the property dictionary, e.g. a
828 title, alternative text or a local language setting.
829 \end{description}
830
831
832 The needed markers can be added with low level code e.g. like this (in pdftex syntax):
833
834
835 \begin{lstlisting}
836 \pdfliteral page {/H1 <</MCID 3>> BDC}%
837 text to be marked
838 \pdfliteral page {EMC}%
839 \end{lstlisting}
840
841
842 This sounds easy. But there are quite a number of traps, mostly with pdfLaTeX:
843
844 \begin{enumerate}[begin-penalty=10000]
845 \item \PDF{} is a page oriented format. And this means that the start
846 \texttt{BDC}/\texttt{BMC} and the corresponding end \texttt{EMC}
847 must be on the same page. So marking e.g. a section title like in the
848 following example won't always work as the literal before the
849 section could end on the previous page:
850
851
852 \begin{lstlisting}
853 \pdfliteral page {/H1 <</MCID 3>> BDC} %problem: possible pagebreak here
854 \section{mysection}
855 \pdfliteral page {EMC}%
856 \end{lstlisting}
857
858
859 Using the literals \emph{inside} the section argument is better, but then
860 one has to take care that they don't wander into the header and the toc.
861
862 \item Literals are \enquote{whatsits} nodes and can change spacing, page
863 and line breaking. The literal \emph{behind} the section in the
864 previous example could e.g. lead to a lonely section title at the end
865 of the page.
866
867 \item The \texttt{/MCID} numbers must be unique on a page. So you can't
868 use the literal in a saved box that you reuse in various places. This
869 is e.\,g. a problem with \texttt{longtable} as it saves the table
870 header and footer in a box.
871
872 \item The \texttt{/MCID}-chunks are leaf nodes in the structure tree, so
873 they shouldn't be nested.
874
875 \item Often text in a document is created automatically or moved around:
876 entries in the table of contents, index, bibliography and more. To
877 mark these text chunks correctly one has to analyze the code creating
878 such content to find suitable places to inject the literals.
879
880
881 \item There exist environments which process their content more than once
882 -- examples are \texttt{align} and \texttt{tabularx}.
883 So one has to check for doublets and holes in the counting system.
884
885
886 \item \PDF{} is a page oriented format. And this means that the start and
887 the end marker must be on the same page \ldots\ \emph{so what to do
888 with normal paragraphs that split over pages??}. This question will
889 be discussed in subsection~\ref{sec:splitpara}.
890 \end{enumerate}
891
892
893 \subsubsection{Generic mode versus Lua mode in the mc-task}
894
895 While in generic mode the commands insert the literals directly and so have
896 all the problems described above the Lua mode works quite differently: The
897 tagging commands don't insert literals but set some (global)
898 \emph{attributes} which are attached to all the following nodes. When the
899 page is shipped out some Lua code is called which wanders through the shipout
900 box and injects the literals at the places where the attributes changes.
901
902 This means that quite a number of problems mentioned above are not relevant
903 for the Lua mode:
904
905 \begin{enumerate}
906 \item Page breaks between start and end of the marker are
907 \emph{not} a problem. So you can mark a complete paragraph. If a pagebreak
908 occur directly after an start marker or before an end marker this can lead to
909 empty chunks in the \PDF{} and so bloat up \PDF{} a bit, but this is imho not
910 really a problem (compared to the size increase by the rest of the tagging).
911
912 \item The commands don't insert literals directly and so affect line and page
913 breaking much less.
914
915 \item The numbering of the MCID are done at shipout, so no label/ref system
916 is needed.
917
918 \item The code can do some marking automatically. Currently everything that
919 has not been marked up by the document is marked as artifact.
920 \end{enumerate}
921
922
923
924
925 \subsubsection{Commands to mark content and chunks}
926
927
928 In generic mode\sidenote{Generic mode only} is vital that the end command is
929 executed on the same page as the begin command. So think carefully how to
930 place them. For strategies how to handle paragraphs that split over pages see
931 subsection~\ref{sec:splitpara}.
932
933 \begin{docCommands}
934 {
935 {doc name=tagmcbegin,doc parameter={\marg{key-val-list}}},
936 {doc name=tag_mc_begin:n,doc parameter={\marg{key-val-list}}}
937 }
938 \end{docCommands}
939
940
941 These commands insert the begin of the marked content code in the \PDF{}.
942 They don't start a paragraph. \emph{They don't start a group}. Such markers
943 should not be nested. The command will warn you if this happens.
944
945 In the generic mode the commands insert literals. These are whatsits and so
946 can affect spacing. In Lua mode they set an attribute \emph{globally}.
947
948 The key-val list understands the following keys:
949
950 \begin{description}
951 \item[\PrintKeyName{tag}] This key is optional. By default the tag name
952 of the surrounding structure is used, which normally should be fine.
953 But if needed the name can be set explicitly with this key. The value
954 of the key is typically one of the standard type listed in section
955 \ref{sec:new-tag} (without a slash at the begin, this is added by the
956 code). It is possible to setup new tags, see the same section. The
957 value of the key is expanded, so it can be a command. The expansion
958 is passed unchanged to the \PDF{}, so it should with a starting slash
959 give a valid \PDF{} name (some ascii with numbers like \texttt{H4}
960 is fine).
961
962 \item[\PrintKeyName{artifact}] This will setup the marked content as an
963 artifact. The key should be used for content that should be ignored.
964 The key can take one of the values \PrintKeyName{pagination},
965 \PrintKeyName{pagination/header}, \PrintKeyName{pagination/footer},
966 \PrintKeyName{layout}, \PrintKeyName{page},
967 \PrintKeyName{background} and \PrintKeyName{notype} (this is the
968 default). Text in the header and footer should normally be marked
969 with \PrintKeyName{artifact=pagination} or
970 \PrintKeyName{pagination/header}, \PrintKeyName{pagination/footer}
971 but simply artifact (as it is now done automatically) should be ok
972 too.
973
974 It is not quite clear if rules and other decorative graphical objects
975 needs to be marked up as artifacts. Acrobat seems not to mind if not, but
976 PAC~3 complained.
977
978 The validators complain if some text is not marked up, but it is not
979 quite clear if this is a serious problem.
980
981 The\sidenote{Lua mode} Lua mode will mark up everything unmarked as
982 \texttt{artifact=notype}. You can suppress this behavior by setting the
983 tagpdfsetup key \texttt{activate/tagunmarked} to false. See section
984 \ref{ssec:setup}.
985
986 \item[\PrintKeyName{stash}] Normally marked content will be stored in the
987 \enquote{current} structure. This may not be what you want. As an
988 example you may perhaps want to put a marginnote behind or before the
989 paragraph it is in the tex-code. With this boolean key the content is
990 marked but not stored in the kid-key of the current structure.
991
992 \item[\PrintKeyName{label}] This key sets a label by which you can call
993 the marked content \emph{later} in another structure (if it has been stashed
994 with the previous key). Internally the label name will start with
995 \texttt{tagpdf-}.
996
997 \item[\PrintKeyName{alt}]
998 This key inserts an \texttt{/Alt} value in the property dictionary
999 of the BDC operator. See section~\ref{sec:alt}.
1000 The value is handled as verbatim string, commands are
1001 not expanded but the value will be expanded first once (so works like
1002 the key \texttt{alttext-o} in previous versions which has been
1003 removed). If the value is empty, nothing will happen.
1004
1005 That means that you can do something like in the following listing
1006 and it will insert \verb+\frac{a}{b}+ (hex encoded) in the \PDF{}.
1007
1008 \begin{lstlisting}
1009 \newcommand\myalttext{\frac{a}{b}}
1010 \tagmcbegin{tag=P,alt=\myalttext}
1011 \end{lstlisting}
1012
1013
1014
1015
1016 \item[\PrintKeyName{actualtext}] This key inserts an \texttt{/ActualText}
1017 value in the property dictionary of the BDC operator. See
1018 section~\ref{sec:alt}. The value is handled as verbatim string,
1019 commands are not expanded but the value will be expanded first once
1020 (so works like the key \texttt{actualtext-o} in previous versions
1021 which has been removed). If the value is empty, nothing will happen.
1022
1023 That means that you can do something like in the following listing and
1024 it will insert \verb+X+ (hex encoded) in the \PDF{}.
1025
1026
1027 \begin{lstlisting}
1028 \newcommand\myactualtext{X}
1029 \tagmcbegin{tag=Span,actualtext=\myactualtext}
1030 \end{lstlisting}
1031
1032 According to the PDF reference, \texttt{/ActualText} should only be used
1033 on marked content sequence of type Span. This is not enforced by the code
1034 currently. There is also some discussion going on, if
1035 \texttt{/ActualText} can actually be used in a MC dictionary or if it
1036 should be in a separate BDC-operator.
1037
1038 \item[\PrintKeyName{lang}] This (experimental) key allows to add a \texttt{/Lang} entry on the MC. This probably is not conforming to the PDF specification but it works in Adobe reader. The value is a BCP-string, e.g. \texttt{en} or \texttt{de}.
1039 The key is not so useful as you typically want to the set `/Lang` entry on all MC inside a region or structure. Once the requirements of the PDF specification has been clarified the interface will be extended.
1040
1041 \item[\PrintKeyName{raw}] This key allows you to add more entries to the
1042 properties dictionary. The value must be correct, low-level \PDF{}.
1043 E.g. \verb+raw=/Alt (Hello)+ will insert an alternative Text.
1044 \end{description}
1045
1046
1047 \begin{docCommands}
1048 {
1049 {doc name=tagmcend},
1050 {doc name=tag_mc_end:}
1051 }
1052 \end{docCommands}
1053
1054 These commands insert the end code of the marked content. They don't end a
1055 group and it doesn't matter if they are in another group as the starting
1056 commands. In generic mode both commands check if there has been a begin
1057 marker and issue a warning if not. In luamode it is often possible to omit
1058 the command, as the effect of the begin command ends with a new
1059 \verb+\tagmcbegin+ anyway.
1060
1061
1062 \begin{docCommands}
1063 {
1064 {doc name=tagmcuse,doc parameter=\marg{label}},
1065 {doc name=tag_mc_use:n,doc parameter=\marg{label}}
1066 }
1067 \end{docCommands}
1068
1069
1070 These commands allow you to record a marked content that you stashed away
1071 into the current structure. Be aware that a marked content can be used only
1072 once -- the command will warn you if you try to use it a second time.
1073
1074
1075 \begin{docCommands}
1076 {
1077 {doc name=tag_mc_end_push:},
1078 {doc name=tag_mc_begin_pop:n,doc parameter=\marg{key-val-list}}
1079 }\end{docCommands}
1080
1081 If there is an open mc chunk,
1082 the first command ends it and pushes its tag on a stack. If there is no
1083 open chunk, it puts $-1$ on the stack (for debugging).
1084 The second command removes a value from the stack. If it is different from
1085 $-1$ it opens a tag with it. The command is mainly meant to be used inside hooks and command
1086 definitions so there is only an expl3 version. Perhaps other content of the mc-dictionary (for example the Lang) needs to be saved on the stacked too.
1087
1088
1089
1090 \begin{docCommands}
1091 {
1092 {doc name=tagmcifinTF,doc parameter=\marg{true code}\marg{false code}},
1093 {doc name=tag_mc_if_in:TF,doc parameter=\marg{true code}\marg{false code}}
1094 }\end{docCommands}
1095
1096
1097 These commands check if a marked content is currently open and allows you to e.g. add the end marker if yes.
1098
1099 In \emph{generic mode}, where marked content command shouldn't be nested, it works with a global boolean.
1100
1101 In \emph{Lua mode} it tests if the mc-attribute is currently unset. You can't test the nesting level with it!
1102
1103
1104
1105 \begin{docCommand}{tag_mc_reset_box:N}{\marg{box}}\end{docCommand}
1106
1107 In Lua mode this command will process the given box and reset all mc
1108 related attributes in the box to the current values. This means that
1109 if the box is used all its contents will be a kid of the current
1110 structure. This should (probably) only be used on boxes which don't
1111 contain tagging commands. See below section~\ref{sec:savebox} for
1112 more details.
1113
1114
1115 \subsubsection{Retrieving data} \label{sec:retrieve}
1116 With more elaborate tagging the need arise to retrieve and store current data.
1117
1118 \begin{docCommand}{tag_get:n}{\marg{key word}}\end{docCommand}
1119
1120 This (expandable) command returns the values of some variables. Currently, the working key words are
1121 \begin{itemize}
1122 \item \verb+mc_tag+: the tag name of the current mc-chunk
1123 \item \verb+struct_tag+: the tag name of the current structure
1124
1125 \item \verb+struct_id+: The ID of the current structure. This is a
1126 string and is returned including parentheses.
1127
1128 \item \verb+struct_num+: This returns a number and works also if only
1129 \pkg{tagpdf-base} has been loaded, but then doesn't give the same
1130 output: if \pkg{tagpdf} is loaded and tagging is active,
1131 \verb+struct_num+ gives the number of currently active structure, so
1132 it reverts to the parent number if a structure is closed. If only
1133 \pkg{tagpdf-base} is loaded nesting of structure is not tracked and
1134 so the command gives back the number of the last structure that has
1135 been created. In luatex this number can also be retrieved with the
1136 Lua function \verb+ltx.tag.get_struct_num()+.
1137
1138 \item \verb+struct_counter+: This returns a number and works also if
1139 only \pkg{tagpdf-base} has been loaded. It gives back the state of
1140 the absolute structure counter and so the number of the last structure
1141 that has been created. This can be used to detect if in a piece of
1142 code there are structure commands. Be aware that this is a \LaTeX{}
1143 counter and so is reset in some places.
1144 In luatex this number can also be retrieved with the
1145 Lua function \verb+ltx.tag.get_struct_counter()+. The number of the next
1146 structure to be created is then \verb+ltx.tag.get_struct_counter()+ increased by
1147 one), this can also be retrieved with the function \verb+ltx.tag.get_struct_num_next()+.
1148
1149 \item \verb+mc_counter+: This returns a number and works also if only
1150 \pkg{tagpdf-base} has been loaded. It gives back the state of the
1151 absolute mc-counter and so number of the last mc-chunk that has been
1152 created. This can be used to detect if in a piece of code there are
1153 mc-commands.
1154 \end{itemize}
1155
1156
1157 \subsubsection{Luamode: global or not global -- that is the question}\label{sec:global-local}
1158
1159 In\sidenote{Lua mode} luamode the mc-commands set and unset an
1160 attribute to mark the nodes. One can view such an attribute like a font
1161 change or a color: they affect all following chars and glue nodes until
1162 stopped.
1163
1164 From version 0.6 to 0.82 the attributes were set locally. This had the
1165 advantage that the attributes didn't spill over in area where they are not
1166 wanted like the header and footer or the background pictures. But it had the
1167 disadvantage that it was difficult for an inner structure to correctly
1168 interrupt the outer mc-chunk if it can't control the group level. For example
1169 this didn't work due to the grouping inserted by the user:
1170
1171 \begin{lstlisting}
1172 \tagstructbegin{tag=P}
1173 \tagmcbegin{tag=P}
1174 Start paragraph
1175 {% user grouping
1176 \tag_mc_end_push:
1177 \tagstructbegin{tag=Em}
1178 \tagmcbegin{tag=Em}
1179 \emph{Emphasized test}
1180 \tagmcend
1181 \tagstructend
1182 \tag_mc_begin_pop:n{}
1183 }% user grouping
1184 Continuation of paragraph
1185 \tagmcend
1186 \tagstructend
1187 \end{lstlisting}
1188
1189 The reading order was then wrong, and the \emph{emphasized text} moved in the structure at the end.
1190
1191 So starting with version 0.9 this has been reverted. The attribute is now global again.
1192 This solves the \enquote{interruption} problem, but has its price: Material inserted by the output routine
1193 must be properly guarded. For example
1194
1195 \begin{lstlisting}
1196 \DocumentMetadata{uncompress}
1197 \documentclass{article}
1198
1199 \pagestyle{headings}
1200 \begin{document}
1201 \sectionmark{HEADER}
1202 \AddToHook{shipout/background}{\put(5cm,-5cm){BACKGROUND}}
1203 \tagmcbegin{tag=P}Page 1\newpage Page 2\tagmcend
1204 \end{document}
1205 \end{lstlisting}
1206
1207 Here the header and the background code on the \emph{first} page will be marked up as paragraph
1208 and added as chunk to the document structure. The header and the background code on
1209 the \emph{second} page will be marked as artifact. The following figure shows how the tags looks
1210 like.
1211
1212 \includegraphics[alt=Show tags of examples]{global-ex}
1213
1214 It is therefore from now on important to correctly markup such code. Header
1215 and footer are now marked as artifacts (see below). If they contain code
1216 which needs a different markup it still must be added explicitly. With
1217 packages like \pkg{fancyhdr} or \pkg{scrlayer-scrpage} it is quite easy to
1218 add the needed code.
1219
1220
1221 \subsubsection{Tips}
1222
1223 \begin{itemize}
1224 \item Mark commands inside floats should work fine (but need perhaps some compilation rounds in generic mode).
1225
1226 \item In case you want to use it inside a \verb+\savebox+ (or some
1227 command that saves the text internally in a box): If the box is used
1228 directly, there is probably no problem. If the use is later, stash
1229 the marked content and add the needed \verb+\tagmcuse+ directly
1230 before or after the box when you use it.
1231
1232 \item Don't use a saved box with markers twice.
1233
1234 \item If boxes are unboxed you will have to analyze the \PDF{} to
1235 check if everything is ok.
1236
1237 \item If you use complicated structures and commands (breakable boxes
1238 like the one from \pkg{tcolorbox}, \pkg{multicol}, many footnotes)
1239 you will have to check the \PDF{}.
1240 \end{itemize}
1241
1242
1243 \begin{figure}
1244
1245 \input{link-figure-input}
1246
1247 \caption{Structure needed for a link annotation}\label{fig:linkannot}
1248
1249 \end{figure}
1250
1251 \subsubsection{Header and Footer}\label{sec:header-footer}
1252
1253 Tagging header and footer is not trivial. At first on the technical side header and footer are
1254 typeset and attached to the page during the output routine and the exact timing is not really under
1255 control of the user. That means that when adding tagging there one has to be careful not to disturb
1256 the tagging of the main text---this is mostly important in luamode where the attributes are global
1257 and can easily spill over.
1258
1259 At second one has to decide about how to tag: in many cases header and footer can simply be ignored,
1260 they only contain information which are meant to visually guide the reader and so are not relevant for
1261 the structure. This means that normally they should be tagged as artifacts. The PDF reference offers
1262 here a rather large number of options here to describe different versions of \enquote{ignore this}.
1263 Typically the header and footer should get the type \texttt{Pagination} and this types has a number of subtypes like
1264 Header, Footer, PageNum. It is not yet known if any technology actually makes use of this info.
1265
1266 But they can also contain meaningful content, for example an address. In such cases the content
1267 should be added to the structure (where?) but even if this address is
1268 repeated on every page at best only once. All this need some thoughts both from the users and the packages and code
1269 providing support for header and footers.
1270
1271 For now tagpdf added some first support for automatically tagging:
1272 Starting with version 0.92 header and footer are by default automatically marked up as (simple) artifacts.
1273
1274 With the key \PrintKeyName{page/exclude-header-footer} the behavior can be
1275 changed: The value \texttt{false} disables the automatic tagging (the resulting tagging is then rather unpredicatable and depends on the surrounding page), the
1276 value \texttt{pagination} adds additionally an \texttt{/Artifact}
1277 structure with the attribute \texttt{/Pagination}, but the contents is still not tagged.
1278
1279 The (experimental) key \PrintKeyName{page/tag-header-footer} surrounds the header with an \texttt{/Artifact} structure too but then reenables tagging. It currently only works if the header and footer contain a \cs{parbox}, so if e.g. the \pkg{fancyhdr} package is used.
1280
1281
1282 If some additional markup (or even a structure) is wanted, something like this should be used (together with \PrintKeyName{page/exclude-header-footer} set to false) (here with
1283 the syntax of the \pkg{fancyhdr} package) to close the open mc-chunk and restart if after the content:
1284
1285 \begin{lstlisting}
1286 \ExplSyntaxOn
1287 \cfoot{\leavevmode
1288 \tag_mc_end_push:
1289 \tagmcbegin{artifact=pagination/footer}
1290 \thepage
1291 \tagmcend
1292 \tag_mc_begin_pop:n{artifact}}
1293 \ExplSyntaxOff
1294 \end{lstlisting}
1295
1296 \subsubsection{Links and other annotations}\label{sec:link+annot}
1297
1298 Annotations (like links or form field annotations) are objects
1299 associated with a geometric region of the page rather than with a
1300 particular object in its content stream. Any connection between a link
1301 or a form field and the text is based solely on visual appearance (the
1302 link text is in the same region, or there is empty space for the form
1303 field annotation) rather than on an explicitly specified association.
1304
1305 To connect such a annotation with the structure and so with
1306 surrounding or underlying text a specific structure has to be added,
1307 see \ref{fig:linkannot}: The annotation is added to a structure
1308 element as an object reference. It is not referenced directly but
1309 through an intermediate object of type OBJR. To the dictionary of the
1310 annotation a \texttt{/StructParent} entry must be added, the value is
1311 a number which is then used in the ParentTree to define a relationship
1312 between the annotation and the parent structure element.
1313
1314 To support this, \pkg{tagpdf} offers currently two commands
1315
1316 \begin{docCommand}{tag_struct_parent_int:}{}\end{docCommand}
1317
1318 This insert the current value of a global counter used to track such
1319 objects. It can be used to add the \texttt{/StructParent} value to the
1320 annotation dictionary.
1321
1322 \begin{docCommand}{tag_struct_insert_annot:nn}{\marg{object reference}\marg{struct parent number}}\end{docCommand}
1323
1324 This will insert the annotation described by the object reference into
1325 the current structure by creating the OBJR object. It will also add
1326 the necessary entry to the parent tree and increase the global counter
1327 referred to by |\tag_struct_parent_int:|. It does nothing if
1328 (structure) tagging is not activated.
1329
1330 Attention! As the second command increases the global counter at the
1331 end it changes the value given back by the first. That means that if
1332 nesting is involved care must be taken that the correct numbers is
1333 used. This should be easy to fulfill for most annotations, as there
1334 are boxes. There the second command should at best be used directly
1335 behind the annotation and it can make use of
1336 |\tag_struct_parent_int:|. For links nesting is theoretically
1337 possible, and it could be that future versions need more sophisticated
1338 handling here.
1339
1340 In environments which process their content twice like tabularx or
1341 align it would be the best to exclude the second command from the
1342 trial step, but this will need better support from these environments.
1343
1344 Typically using this commands is not often needed: Since version 0.81
1345 \pkg{tagpdf} already handles (unnested) links, and form fields created
1346 with the \pkg{l3pdffield} package will be handled by this
1347 package.
1348
1349 The following listing shows low-level to create link where the two
1350 commands are used:
1351
1352
1353 \begin{lstlisting}
1354 \pdfextension startlink
1355 attr
1356 {
1357 /StructParent \tag_struct_parent_int: %<----
1358 }
1359 user {
1360 /Subtype/Link
1361 /A
1362 <<
1363 /Type/Action
1364 /S/URI
1365 /URI(http://www.dante.de)
1366 >>
1367 }
1368 This is a link.
1369 \pdfextension endlink
1370 \tag_struct_insert_annot:xx {\pdfannot_link_ref_last:}{\tag_struct_parent_int:}
1371 \end{lstlisting}
1372
1373
1374
1375 \subsubsection{Math}
1376
1377 Math is still a problem but some progress has been made.
1378 To tag math you have to surround it with a \texttt{Formula} structure. But the content of such a structure is handled by readers as a black box so additional data is needed for accessibility.
1379
1380 There are a number of theoretical options here:
1381 \begin{enumerate}
1382 \item One can add an alternative text (\texttt{/Alt}) or an \texttt{/ActualText}
1383 to the structure element either some text manually provided by the author or (with
1384 the math module in the latex-lab bundle) the \LaTeX-source).
1385 \item One can add an alternative text (\texttt{/Alt} or \texttt{/ActualText})
1386 to the MC-chunks.
1387 \item One can build inside the \texttt{Formula} structure element a tree with MathML structure elements --- with PDF 2.0 this not require to declare new tags as the MathML name space is built-in.
1388 \item One can in PDF 2.0 attach a MathML file and/or the \LaTeX-source as associated file to the \texttt{Formula} structure (or to one or more MC-chunks).
1389 \end{enumerate}
1390 The question is how these work in reality.
1391
1392 Option 1 and 2 give not too bad results
1393 with a screen reader, but can
1394 require manual work and if you are unlucky the reader drops
1395 important part of the math (like punctuation symbols).
1396 Exploring the equation is not possible.
1397
1398
1399 Option 3 creates many structure elements.
1400 E.g. I have seen an example where \emph{every single
1401 symbol} has been marked up with tags from MathML along with an
1402 \texttt{/ActualText} entry and an entry with alternate text which
1403 describes how to read the symbol. The \PDF{} then looked like this
1404
1405
1406 \begin{lstlisting}
1407 /mn <</MCID 6 /ActualText<FEFF0034>/Alt( : open bracket: four )>>BDC
1408 ...
1409 /mn <</MCID 7 /ActualText<FEFF0033>/Alt( third s )>>BDC
1410 ...
1411 /mo <</MCID 8 /ActualText<FEFF2062>/Alt( times )>>BDC
1412 \end{lstlisting}
1413
1414
1415 If this is really the way to go one would need some script to add the
1416 mark-up as doing it manually is too much work and would make the
1417 source unreadable -- at least with pdflatex and the generic mode. In
1418 Lua mode is it possible to hook into the \texttt{mlist\_to\_hlist}
1419 callback and add marker automatically. Some first implementation in this direction
1420 has been done by Marcel Krüger in the luamml project. But up-to-now it was not possible
1421 to test the usability of this approach: With the exception of the html derivation
1422 with ngpdf no PDF-viewer/screen reader combination
1423 seems to make use of such structures.
1424 I'm not sure anyway that this is the best way to do math. It looks rather
1425 odd that a document should have to tell a screen reader in such detail
1426 how to read an equation.
1427
1428 The last option 4 has been implemented in the math module in the \texttt{latex-lab}
1429 bundle. Here happily a proof of
1430 concept was possible: With development versions of foxit and the NVDA reader
1431 it was possible to access an attached MathML and get speech output from it \cite{todasoifferdeims2024,mittelbachfischerdeims2024}. See also \cite{mathexamples} for some
1432 examples and section~\ref{sec:alt} for some more remarks and tests.
1433
1434
1435 \subsubsection{Split paragraphs}\label{sec:splitpara}
1436
1437 %TODO: think about marginnote! Aside?
1438
1439 A\sidenote{Generic mode only} problem in generic mode are paragraphs
1440 with page breaks. As already mentioned the end marker \texttt{EMC}
1441 must be added on the same page as the begin marker. But it is in
1442 pdflatex \emph{very} difficult to inject something at the page break
1443 automatically. One can manipulate the shipout box to some extend in
1444 the output routine, but this is not easy and it gets even more
1445 difficult if inserts like footnotes and floats are involved: the end
1446 of the paragraph is then somewhere in the middle of the box.
1447
1448 So with pdflatex in generic mode one until now had to do the splitting manually.
1449
1450 The example \texttt{mc-manual-para-split} demonstrates how this can be
1451 done. The general idea was to use \verb+\vadjust+ in the right place:
1452
1453
1454 \begin{lstlisting}
1455 \tagmcbegin{tag=P}
1456 ...
1457 fringilla, ligula wisi commodo felis, ut adipiscing felis dui in
1458 enim. Suspendisse malesuada ultrices ante.% page break
1459 \vadjust{\tagmcend\pagebreak\tagmcbegin{tag=P}}
1460 Pellentesque scelerisque
1461 ...
1462 sit amet, lacus.\tagmcend
1463 \end{lstlisting}
1464
1465 Starting with version 0.92 there is code which resolves this
1466 problem. Basically it works like this: every mc-command issues a mark
1467 command (actually two slightly different). When the page is built in
1468 the output routine this mark commands are inspected and from them
1469 \LaTeX{} can deduce if there is a mc-chunk which must be closed or
1470 reopened. The method is described in Frank Mittelbach's talk at
1471 TUG~2021 \enquote{Taming the beast — Advances in paragraph tagging
1472 with pdfTeX and XeTeX} \url{https://youtu.be/SZHIeevyo3U?t=19551}.
1473
1474
1475 Please note
1476
1477 \begin{itemize}
1478 \item Typically you will need more compilations than previously, don't
1479 rely on the rerun messages, but if something looks wrong rerun.
1480
1481 \item The code relies on that related |\tagmcbegin| and |\tagmcend|
1482 are in the same boxing level. If one is in a box (which hides the
1483 marks) and the other in the main galley, things will go wrong (\texttt{longtable}
1484 is for example problematic).
1485 \end{itemize}
1486
1487 \subsubsection{Automatic tagging of paragraphs}\label{sec:paratagging}
1488
1489 Another feature that emerged from the \LaTeX{} tagged PDF project are hooks
1490 at the begin and end of paragraphs. The tagging code makes use of these hooks to
1491 tag paragraphs. In the first version it added only one structure, but this
1492 proved to be not adequate:
1493
1494 Paragraphs in \LaTeX{} can be nested, e.g., you can have a paragraph
1495 containing a display quote, which in turn consists of more than one
1496 (sub)paragraph, followed by some more text which all belongs to the
1497 same outer paragraph.
1498
1499 In the \PDF{} model and in the HTML model that is not supported: the rules in
1500 \PDF{} specification do not allow \texttt{P}-structures to be nested --- a
1501 limitation that conflicts with real live, given that such constructs are
1502 quite normal in spoken and written language.
1503
1504 The approach we take (starting with march 2023) to resolve
1505 this is to model such \enquote{semantic} paragraphs with a structure named
1506 \texttt{\UseStructureName{para/semantic}} and use \texttt{P}
1507 (under the name \texttt{\UseStructureName{para/textblock}}) only for
1508 (portions of) the actual paragraph text in a way that the \texttt{P}s are not
1509 nested. As a result we have for a simple paragraph two structures:
1510
1511 \begin{lstlisting}
1512 <text-unit>
1513 <P>
1514 The paragraph text ...
1515 </P>
1516 </text-unit
1517 \end{lstlisting}
1518
1519 In the case of an element, such as a display quote or a display list
1520 inside the paragraph, we then have
1521 \begin{lstlisting}
1522 <text-unit>
1523 <text>
1524 The paragraph text before the display element ...
1525 </text>
1526 <display element structure>
1527 Content of the display structure possibly involving inner <text-unit> tags
1528 </display element structure>
1529 <text>
1530 ... continuing the outer paragraph text
1531 </text>
1532 </text-unit>
1533 \end{lstlisting}
1534
1535
1536 In other words such a display block is always embedded in a |<text-unit>|
1537 structure, possibly preceded by a |<text>|\ldots|</text>| block and possibly
1538 followed by one, though both such blocks are optional. More information about
1539 this can be found in the documentation of \texttt{latex-lab-block}.
1540
1541 As a consequence the tagging code adds two structures if paratagging is
1542 activated. The code to tag display blocks extends this code to handle the
1543 nesting of lists and other display structures.
1544
1545 The automatic tagging require that for every begin of a paragraph with the
1546 begin hook code there a corresponding end with the closing hook code. This
1547 can fail, e.g if a |vbox| doesn't correctly issue a |\par| at the end. If
1548 this happens the tagging structure can get very confused. At the end of the
1549 document \pkg{tagpdf} checks if the number of outer and inner start and end
1550 paragraph structures created with the automatic paratagging code are equal
1551 and it will error if not.
1552
1553 The automatic tagging of paragraphs can be deactivated completely or only the
1554 outer level with the |\tagpdfsetup| key |para/tagging| and |para-flattened|
1555 or with the commands |\tagpdfparaOn| and |\tagpdfparaOff| or by changing the plugs
1556 of the tagging sockets -- they are defined and described in source2e, section lttagging.
1557
1558
1559 Nesting the activation and deactivation of the tagging of paragraphs can be
1560 quite difficult. For example if it is unclear if the inner code issues a
1561 |\par| or not it is not trivial to exclude an end hook for every excluded
1562 begin hook. In such cases it can be easier to use the |para/tag| key with the
1563 value |NonStruct| to convert some |P|-structures into |NonStruct|-structures
1564 without real meaning.
1565
1566 \subsection{Task 2: Marking the structure}
1567
1568 The structure is represented in the \PDF{} with a number of objects of type
1569 \texttt{StructElem} which build a tree: each of this objects points back to
1570 its parent and normally has a number of kid elements, which are either again
1571 structure elements or -- as leafs of the tree -- the marked contents chunks
1572 marked up with the \verb+tagmc+-commands. The root of the tree is the
1573 \texttt{StructTreeRoot}.
1574
1575
1576
1577 \subsubsection{Structure types}
1578
1579 The tree should reflect the \emph{semantic} meaning of the text. That means
1580 that the text should be marked as section, list, table head, table cell and
1581 so on. A number of standard structure types is predefined, see section
1582 \ref{sec:new-tag} but it is allowed to create more. If you want to use types
1583 of your own you must declare them. E.g. this declares two new types
1584 \texttt{TAB} and {FIG} and bases them on \texttt{P}:
1585
1586
1587 \begin{lstlisting}
1588 \tagpdfsetup{
1589 role/new-tag = TAB/P,
1590 role/new-tag = FIG/P,
1591 }
1592 \end{lstlisting}
1593
1594
1595
1596 \subsubsection{Sectioning}
1597
1598 The sectioning units can be structured in two ways: a flat, html-like and a
1599 more (in pdf/UA2 basically deprecated) xml-like version. The flat version
1600 creates a structure like this:
1601
1602
1603 \begin{lstlisting}
1604 <H1>section header</H1>
1605 <P> text</P>
1606 <H2>subsection header</H2>
1607 ...
1608 \end{lstlisting}
1609
1610
1611 So here the headings are marked according their level with \texttt{H1}, \texttt{H2}, etc.
1612
1613
1614 In the xml-like tree the complete text of a sectioning unit is surrounded
1615 with the \texttt{Sect} tag, and all headers with the tag \texttt{H}. Here the
1616 nesting defines the level of a sectioning heading.
1617
1618
1619 \begin{lstlisting}
1620 <Sect>
1621 <H>section heading</H>
1622 <P> text</p>
1623 <Sect>
1624 <H>subsection heading</H>
1625 ...
1626 </Sect>
1627 </Sect>
1628 \end{lstlisting}
1629
1630
1631
1632 The flat version is more \LaTeX-like and it is rather straightforward to
1633 patch \verb+\chapter+, \verb+\section+ and so on to insert the appropriates
1634 \texttt{H\ldots} start and end markers. The xml-like tree is more difficult
1635 to automate. It has been implemented in the sec module in latex-lab, but can break
1636 if sectioning commands are hidden inside boxes.
1637
1638
1639
1640 \subsubsection{Commands to define the structure}
1641
1642
1643 The following commands can be used to define the tree structure:
1644
1645
1646 \begin{docCommands}
1647 {
1648 {doc name=tagstructbegin,doc parameter=\marg{key-val-list}},
1649 {doc name=tag_struct_begin:n,doc parameter=\marg{key-val-list}}
1650 }\end{docCommands}
1651
1652 These commands start a new structure. They don't start a group. They set all their values globally.
1653
1654 The key-val list understands the following keys:
1655
1656 \begin{description}
1657 \item[\PrintKeyName{tag}] This is required. The value of the key is
1658 normally one of the standard types listed in section
1659 \ref{sec:new-tag}. It is possible to setup new tags/types, see the
1660 same section. The value can also be of the form |type/NS|, where
1661 |NS| is the shorthand of a declared name space. Currently the
1662 names spaces |pdf|, |pdf2|, |mathml| and |user| are defined. This
1663 allows to use a different name space than the one connected by
1664 default to the tag. But normally this should not be needed.
1665
1666 \item[\PrintKeyName{stash}] Normally a new structure inserts itself
1667 as a kid into the currently active structure. This key prohibits
1668 this. The structure is nevertheless from now on \enquote{the
1669 current active structure} and parent for following marked
1670 content and structures.
1671
1672 \item[\PrintKeyName{label}] This key sets a label by which one can
1673 refer to the structure. Currently the key writes a property whose
1674 name starts with \texttt{tagpdfstruct-} to the aux-file with the two
1675 attributes \texttt{tagstruct} (the structure number) and
1676 \texttt{tagstructobj} (the object reference) but also stores the
1677 name and the structure number into a prop for use in the current compilation.
1678 The label is e.g. used by \cs{tag\_struct\_use:n} and by the |ref|
1679 key (which can refer to future structures).
1680
1681 \item[\PrintKeyName{parent}] With the parent key one can choose another
1682 parent. The value is a structure number which must refer to an
1683 already existing, previously created structure. Such a structure
1684 number can have been stored previously with \cs{tag\_get:n}, but one
1685 can also use a label on the parent structure and then use
1686 \cs{property\_ref:nn}\verb+{tagpdfstruct-label}{tagstruct}+ to retrieve
1687 it.
1688
1689 \item[\PrintKeyName{firstkid}] If this key is used the structure is
1690 added at the left of the kids of the parent structure (if the structure is not stashed). This means that it will be the first kid of the structure (unless some
1691 later structure uses the key too). This can be needed e.g. for a caption as
1692 the PDF reference requires it to be the first or last kid of its structure.
1693
1694 \item[\PrintKeyName{alt}] This key inserts an \texttt{/Alt} value in the
1695 dictionary of structure object, see section~\ref{sec:alt}. The value
1696 is handled as verbatim string and hex encoded. The value will be
1697 expanded first once (so works like the key \texttt{alttext-o} in
1698 previous versions which has been removed). If the value is empty,
1699 nothing will happen.
1700
1701 That means that you can do something like this:
1702
1703
1704 \begin{lstlisting}
1705 \newcommand\myalttext{\frac{a}{b}}
1706 \tagstructbegin{tag=P,alt=\myalttext}
1707 \end{lstlisting}
1708
1709
1710
1711 and it will insert \verb+\frac{a}{b}+ (hex encoded) in the
1712 \PDF{}. In case that the text begins with a command that should not
1713 be expanded protect it e.g. with a \verb+\empty+.
1714
1715
1716 \item[\PrintKeyName{actualtext}] This key inserts an \texttt{/ActualText}
1717 value in the dictionary of structure object, see
1718 section~\ref{sec:alt}. The value is handled as verbatim string. The
1719 value will be expanded first once (so works like the key
1720 \texttt{alttext-o} in previous versions which has been removed). If
1721 the value is empty, nothing will happen.
1722
1723 That means that you can do something like this:
1724
1725 \begin{lstlisting}
1726 \newcommand\myactualtext{X}
1727 \tagstructbegin{tag=P,actualtext=\myactualtext}
1728 \end{lstlisting}
1729
1730
1731
1732 and it will insert \verb+X+ (hex encoded) in the \PDF{}. In case
1733 that the text begins with a command that should not be expanded
1734 protect it e.g. with a \verb+\empty+
1735
1736 \item[\PrintKeyName{attribute}] This key takes as argument a comma
1737 list of attribute names (use braces to protect the commas from
1738 the external key-val parser) and allows to add one or more
1739 attribute dictionary entries in the structure object. As an
1740 example
1741
1742 \begin{lstlisting}
1743 \tagstructbegin{tag=TH,attribute= TH-row}
1744 \end{lstlisting}
1745
1746 See also section~\ref{sec:attributes}.
1747
1748 \item[\PrintKeyName{attribute-class}] This key takes as argument a
1749 comma list of attribute names (use braces to protect the commas
1750 from the external key-val parser) and allows to add them as
1751 attribute classes to the structure object. As an example
1752
1753
1754 \begin{lstlisting}
1755 \tagstructbegin{tag=TH,attribute-class= TH-row}
1756 \end{lstlisting}
1757
1758 See also section~\ref{sec:attributes}.
1759
1760 \item[\PrintKeyName{title}] This key allows to set the dictionary
1761 entry \texttt{/T} (for a title) in the structure object. The value
1762 is handled as verbatim string and hex encoded. Commands are not
1763 expanded.
1764
1765 \item[\PrintKeyName{title-o}] This key allows to set the dictionary
1766 entry \texttt{/T} in the structure object. The value is expanded
1767 once and then handled as verbatim string like the
1768 \PrintKeyName{title} key.
1769
1770 \item[\PrintKeyName{AF}] This key allows to reference an associated
1771 file in the structure element. The value should be the name of
1772 an object pointing to the \texttt{/Filespec} dictionary as
1773 expected by \verb+\pdf_object_ref:n+ from a current
1774 \texttt{l3kernel}. For example:
1775
1776 \begin{lstlisting}
1777 \group_begin:
1778 \pdfdict_put:nnn {l_pdffile/Filespec} {AFRelationship}{/Supplement}
1779 \pdffile_embed_file:nnn{example-input-file.tex}{}{tag/AFtest}
1780 \group_end:
1781 \tagstructbegin{tag=P,AF=tag/AFtest}
1782 \end{lstlisting}
1783
1784 As shown, the wanted AFRelationship can be set by filling the dictionary
1785 with the value. The mime type is here detected automatically, but for
1786 unknown types it can be set too. See the \texttt{l3pdffile}
1787 documentation for details. Associated files are a concept new in PDF
1788 1.7, but the code currently doesn't check the pdf version, it is your
1789 responsibility to set it (this can be done with the \texttt{pdfversion}
1790 key in \verb+\DocumentMetadata+).
1791
1792 \item[\PrintKeyName{root-AF}] This key allows to reference an
1793 associated file in the root structure element. The use of the key is no longer usefull as ngpdf \cite{ngpdf} will change its implementation and use files attached to the Catalog.
1794
1795 \item[\PrintKeyName{root-supplemental-file}] This is a variant of the previous key. It takes as argument a file name. It then embeds this file with \texttt{/AFRelationship /Supplement} and appends it as associated file to the structure root. This key is like \texttt{root-AF} no longer recommended. Use the following key instead.
1796
1797 \item[\PrintKeyName{catalog-supplemental-file}] This key should be used instead of \texttt{root-supplemental-file}. It takes as argument a file name. It then embeds this file with \texttt{/AFRelationship /Supplement} and appends it as associated file to the Catalog. ngpdf \cite{ngpdf} will store a \texttt{.css} attached in this way and reference it in the head of the html. If a \texttt{html} is attached in this way, ngpdf will copy the content into the head of the derived html. This means that the content of such an html file should normally be some html snippet suitable for the head, e.g. some css-code inside \texttt{<style>} tags.
1798
1799
1800 \item[\PrintKeyName{AFinline}] This key allows to embed an
1801 associated file with inline content. The value is some text,
1802 which is embedded in the PDF as a text file with mime type
1803 text/plain.
1804
1805 \begin{lstlisting}
1806 \tagstructbegin{tag=P,AFinline=Some extra text}
1807 \end{lstlisting}
1808
1809 \item[\PrintKeyName{AFinline-o}]
1810 This is like \verb+AFinline+, but it expands the value once.
1811
1812 \item[\PrintKeyName{texsource}] This is like \verb+AFinline-o+, but
1813 it creates a tex-file, with mime type \texttt{application/x-tex}
1814 and the AFRelationship \texttt{Source}. It also sets the /Desc key to a
1815 (currently) fix text to satisfy some validators.
1816
1817 \item[\PrintKeyName{mathml}] This is like \verb+AFinline-o+, but it
1818 creates a xml-file, with mime type \texttt{application/xml} and
1819 the AFRelationship \texttt{Supplement}. It also sets the /Desc key to a
1820 (currently) fix text to satisfy some validators.
1821
1822 \item[\PrintKeyName{lang}] This key allows to set the language for
1823 a structure element. The value should be a bcp-identifier,
1824 e.g. |de-De|. It can also be set \enquote{from the outside} for all
1825 structures in the current group with \cs{tagpdfsetup} and the |text/lang| key.
1826
1827 \item[\PrintKeyName{ref}] This key allows to add references to
1828 other structure elements, it adds the |/Ref| array to the
1829 structure. The value should be a comma separated list of
1830 structure labels set with the |label| key.
1831 e.g. |ref={label1,label2}|. It can be used more than once in the
1832 key/value argument and combines the references. See below
1833 in section~\ref{sec:Refkey} for an extended discussion about the |/Ref|
1834 array.
1835
1836 \item[\PrintKeyName{E}] This key sets the |/E| key, the expanded
1837 form of an abbreviation or an acronym (I couldn't think of a
1838 better name, so I sticked to E).
1839
1840 \item[\PrintKeyName{phoneme}] This key sets the |/Phoneme| key, a pronunciation hint. The value should be a IPA text (the PDF spec
1841 allows other variants but this not yet implemented).
1842 This is a new key from PDF 2.0 but as far as I know not used by any
1843 viewer.
1844 \end{description}
1845
1846 \begin{docCommands}
1847 { {doc name=tagstructend}, {doc name=tag_struct_end:}}
1848 \end{docCommands}
1849
1850
1851 These commands end a structure. They don't end a group and it doesn't
1852 matter if they are in another group as the starting commands.
1853
1854 \begin{docCommands}
1855 {
1856 {doc name=tagstructuse,doc parameter=\marg{label}},
1857 {doc name=tag_struct_use:n,doc parameter=\marg{label}}
1858 }\end{docCommands}
1859
1860 These commands insert a structure previously stashed away as kid into the
1861 currently active structure. A structure should be used only once, if the
1862 structure already has a parent you will get a warning.
1863
1864 \subsubsection{Updating structure keys}\label{sec:Refkey}
1865
1866 In a number of cases structures must be updated later. Such updates often can not simply replace values but must extend them in special ways. For this a generic commands exists. Currently it supports updating the \texttt{/A} key (attributes) and the \texttt{Ref} key.
1867
1868 \begin{docCommands}
1869 {
1870 {doc name=tag_struct_gput:nnn,doc parameter=\marg{structurenumber}\marg{keyword}\marg{value}},
1871 }\end{docCommands}
1872
1873 The allowed \meta{keywords} are \texttt{ref}, \texttt{ref\_label} \texttt{ref\_dest},
1874 \texttt{ref\_dest\_parent},\texttt{ref\_num} and \texttt{attribute}.
1875
1876
1877 This command allows to add the value, the target structure of the \texttt{Ref} key, with four methods: directly as object reference (key word \texttt{ref}), through a label name set with the |label| key (key word \texttt{ref\_label}), through a destination name if a \cs{MakeLinkTarget} has been used in
1878 the target structure (key words \texttt{ref\_dest} and
1879 \texttt{ref\_dest\_parent})---this also works if hyperref has not been loaded---and through
1880 the structure number, which has been stored e.g. in a label.
1881
1882 With the keyword \texttt{attribute} the attribute of the structure is
1883 extended. The value is the content of one attribute dictionary, so for example \texttt{/O /Layout /BBox [10 10 50 50]}.
1884
1885 \subsubsection{Root structure}
1886
1887 A document should have at least one structure which contains the whole
1888 document. A suitable tag is \texttt{Document}. Such a root is now always
1889 added automatically. Its type can be changed with the key
1890 \texttt{activate}.
1891
1892 \subsubsection{Attributes and attribute classes}\label{sec:attributes}
1893
1894
1895 Structure Element can have so-called attributes. A single attribute is a
1896 dictionary (or a stream but this is currently not supported by the package
1897 as I don't know an use-case) with at least the required key \verb+/O+ (for
1898 \enquote{Owner} which describes the scope the attribute applies too. As an
1899 example here an attribute that can be attached to tabular header (type TH)
1900 and adds the info that the header is a column header:
1901
1902 \begin{lstlisting}
1903 <</O /Table /Scope /Column>>
1904 \end{lstlisting}
1905
1906
1907 One or more such attributes can be attached to a structure element. It is
1908 also possible to store such an attribute under a symbolic name in a
1909 so-called \enquote{ClassedMap} and then to attach references to such
1910 classes to a structure.
1911
1912
1913 To use such attributes you must at first declare it in \verb+\tagpdfsetup+
1914 with the key \texttt{role/new-attribute}. This key takes two argument, a name and
1915 the content of the attribute. The name should be a sensible key name, it is
1916 converted to a pdf name with \verb+\pdf_name_from_unicode_e:n+, so slashes
1917 and spaces are allow. The content should be a dictionary without the
1918 bracket.
1919
1920
1921 \begin{lstlisting}
1922 \tagpdfsetup
1923 {
1924 role/new-attribute =
1925 {TH-col}{/O /Table /Scope /Column},
1926 role/new-attribute =
1927 {TH-row}{/O /Table /Scope /Row},
1928 }
1929 \end{lstlisting}
1930
1931
1932 Attributes are only written to the \PDF{} when used, so it is not a problem
1933 to predeclare a number of standard attributes.
1934
1935
1936 It is your responsibility that the content of the dictionary is valid
1937 \PDF{} and that the values are sensible!
1938
1939
1940 Attributes can then be used with the key \PrintKeyName{attribute} or
1941 \PrintKeyName{attribute-class} which both take a comma list of attribute
1942 names as argument:
1943
1944
1945 \begin{lstlisting}
1946 \tagstructbegin{tag=TH,
1947 attribute-class= {TH-row,TH-col},
1948 attribute = {TH-row,TH-col},
1949 }
1950 \end{lstlisting}
1951
1952 \subsubsection{Checking the structure}
1953
1954 \begin{docCommands}
1955 {
1956 {doc name=tag_if_in:nTF,doc parameter=\marg{structure tag}},
1957 }\end{docCommands}
1958
1959 With this command it is possible to test if a structure with tag \marg{structure tag} is a (direct or indirect) parent. \marg{structure tag} must be a standard structure type, no role mapping is applied.
1960
1961 \subsection{Task 3: tree Management}
1962
1963 When all the document content has been correctly marked and the data for
1964 the trees has been collected they must be flushed to the \PDF{}. This is
1965 done automatically (if the package has been activated) with an internal
1966 command in an end document hook.
1967
1968 \begin{docCommand}{__tag_finish_structure:}{}\end{docCommand}
1969
1970 This will hopefully write all the needed objects and values to the \PDF{}.
1971 (Beside the already mentioned \texttt{StructTreeRoot} and
1972 \texttt{StructElem} objects, additionally a so-called \texttt{ParentTree}
1973 is needed which records the parents of all the marked contents bits, a
1974 \texttt{Rolemap}, perhaps a \texttt{ClassMap} and object for the
1975 attributes, and a few more values and dictionaries).
1976
1977 \subsection{A fully marked up document body}
1978
1979
1980 The following shows the marking needed for a section, a sentence and a list
1981 with two items. It is obvious that one wouldn't like to have to do this for
1982 real documents. If tagging should be usable, the commands must be hidden as
1983 much as possible inside suitable \LaTeX\ commands and environments.
1984
1985
1986 \begin{lstlisting}
1987 \begin{document}
1988
1989 \tagstructbegin{tag=Document}
1990
1991 \tagstructbegin{tag=Sect}
1992 \tagstructbegin{tag=H}
1993 \tagmcbegin{tag=H} %avoid page break!
1994 \section{Section}
1995 \tagmcend
1996 \tagstructend
1997 \tagstructbegin{tag=P}
1998 \tagmcbegin{tag=P,raw=/Alt (x)}
1999 a paragraph\par x
2000 \tagmcend
2001 \tagstructend
2002
2003 \tagstructbegin{tag=L} %List
2004 \tagstructbegin{tag=LI}
2005 \tagstructbegin{tag=Lbl}
2006 \tagmcbegin{tag=Lbl}
2007 1.
2008 \tagmcend
2009 \tagstructend
2010 \tagstructbegin{tag=LBody}
2011 \tagmcbegin{tag=P}
2012 List item body
2013 \tagmcend
2014 \tagstructend %lbody
2015 \tagstructend %Li
2016
2017 \tagstructbegin{tag=LI}
2018 \tagstructbegin{tag=Lbl}
2019 \tagmcbegin{tag=Lbl}
2020 2.
2021 \tagmcend
2022 \tagstructend
2023 \tagstructbegin{tag=LBody}
2024 \tagmcbegin{tag=P}
2025 another List item body
2026 \tagmcend
2027 \tagstructend %lbody
2028 \tagstructend %Li
2029 \tagstructend %L
2030
2031 \tagstructend %Sect
2032 \tagstructend %Document
2033 \end{document}
2034 \end{lstlisting}
2035
2036 \subsection{Interrupting the tagging}
2037
2038
2039 Experience showed that it must be possible to interrupt tagging in some
2040 places. For example various packages do trial typesetting to measure text
2041 and this shouldn't create structures. There are therefore a number of
2042 commands for various use cases\footnote{it is quite possible that some of
2043 the commands will disappear again if we realize that they are not fitting!}
2044
2045 Warning! Stopping tagging should be done only with care and when it is
2046 ensured that no code inside the stopped part gets confused. Most importantly
2047 currently tagging should not be stopped if a page break can occur
2048 or the output routine is called.
2049
2050 \begin{docCommands}
2051 {
2052 {doc name=tag_suspend:n,doc parameter=\marg{label}},
2053 {doc name=tag_resume:n,doc parameter=\marg{label}}
2054 }
2055 \end{docCommands}
2056
2057 These commands suspend and resume tagging in the current group by switching
2058 \emph{local} booleans. They also stop the increasing of the counters which
2059 keep track of paragraphs if the correct wrapper commands are used.
2060
2061 Restarting tagging is normally only needed if groups can't be used and then
2062 must be done with care: |\tag_resume:n| should normally only restart tagging if
2063 the corresponding stop command actually stopped tagging. This is implement
2064 through a local counter which keeps track of the level.
2065
2066 The \meta{label} can be used to identify the command in debugging message. The label is not expanded and so can for example be a single command token.
2067
2068 The commands are the L3-layer versions of |\SuspendTagging| and |\ResumeTagging|
2069 and will be available in the kernel with the 2024 november release.
2070
2071 \begin{lstlisting}
2072 \tag_suspend:n{\outercommand}
2073 ...
2074 \tag_suspend:n{\innercommand}
2075 ...
2076 \tag_resume:n{\innercommand}
2077 ...
2078 \tag_resume:n{\outercommand}
2079 \end{lstlisting}
2080
2081 \begin{docCommands}
2082 {
2083 {doc name=tag_stop:},
2084 {doc name=tag_start:},
2085 {doc name=tagstop},
2086 {doc name=tagstart},
2087 {doc name=tag_stop:n,doc parameter=\marg{label}},
2088 {doc name=tag_start:n,doc parameter=\marg{label}}
2089 }
2090 \end{docCommands}
2091
2092 These commands are now deprecated in favor or |\tag_suspend:n| and |\tag_resume:n|
2093 but are still provided for some time.
2094
2095
2096
2097
2098 \subsection{Adding tagging to commands}
2099
2100
2101 As mentioned above the mc-markers should not be nested. Basically you write:
2102
2103
2104 \begin{lstlisting}
2105 \tagmcbegin{..}some text ...\tagmcend
2106 <optional structure commands>
2107 \tagmcbegin{..}some other text\tagmcend
2108 \end{lstlisting}
2109
2110
2111 This is quite workable as long as you mark everything manually. But
2112 when defining commands you have to ensure that they correctly push and
2113 pop the mc-chunks where needed.
2114
2115
2116 \section{Alternative text, ActualText and text-to-speech software}\label{sec:alt}
2117
2118
2119
2120 The \PDF{} format allows to add alternative text through the
2121 \PrintKeyName{/Alt} and the \PrintKeyName{/ActualText} key. Both can
2122 be added either to the marked content in the page stream or to the
2123 object describing the structure.
2124
2125
2126 The value of \PrintKeyName{/ActualText} (inserted by \texttt{tagpdf}
2127 with \PrintKeyName{actualtext}) is meant to replace single characters
2128 or rather small pieces of text. It can be used also without any
2129 tagging (e.g. with the package accsupp). If the \PDF{} reader support
2130 this (adobe reader does, sumatra not) one can change with it how a
2131 piece of text is copied and pasted e.g. to split up a ligature.
2132
2133 \PrintKeyName{/Alt} (inserted by \texttt{tagpdf} with
2134 \PrintKeyName{alt}) is a key to improve accessibility: with it one can
2135 add to a picture or something else an alternative text.
2136
2137 The file \texttt{ex-alt-actualtext.tex} shows some experiments I made
2138 with both keys and text-to-speech software (the in-built of adobe and
2139 nvda). To sum them up:
2140
2141 \begin{itemize}
2142 \item The keys have an impact on text-to-speech software only if the
2143 document is fully tagged.
2144
2145 \item \PrintKeyName{/ActualText} should be at best used around short
2146 pieces of marked content.
2147
2148 \item \PrintKeyName{/Alt} is used at best with a structure -- this
2149 avoids problems with luatex where marked contents blocks can be
2150 split over pages.
2151
2152 \item To some extend one can get a not so bad reading of math with the
2153 alternative text.
2154 \end{itemize}
2155
2156
2157
2158
2159 \section{Standard types and new tags}\label{sec:new-tag}
2160
2161 The tags used to describe the type of a structure element can be
2162 rather freely chosen. PDF 1.7 and earlier only requires that in a
2163 tagged PDF all types should be either from a known set of standard
2164 types or are \enquote{role mapped} to such a standard type. Such a
2165 role mapping is a simple key-value in the RoleMap dictionary.
2166
2167 So instead of |H1| the type |section| could be used. The role mapping
2168 can then be declared with the |role/new-tag| key:
2169
2170 \begin{lstlisting}
2171 \tagpdfsetup{role/new-tag = section/H1}
2172 \end{lstlisting}
2173
2174 In PDF 2.0 the situation is a bit more complicated. At first PDF~2.0
2175 introduced \emph{name spaces}. That means that a type can have more
2176 than one \enquote{meaning} depending on the name space it belongs to.
2177 |section (name space A)| and |section (name space B)| are two
2178 different types.
2179
2180 At second PDF 2.0 still requires that a tagged PDF maps all types to a
2181 standard type, but now there are three sets of standard types (The meanings
2182 of the PDF types can be looked up in the \PDF{}-references
2183 \parencite{pdfspec-iso32000-1,pdfspec-iso32000-2_2020}):
2184
2185 \begin{enumerate}
2186 \item The \emph{standard structure namespace for PDF 1.7}, also called
2187 the \emph{default standard structure namespace}. The public name of
2188 the namespace is |tag/NS/pdf|. This can be used to reference the
2189 namespace e.g. in attributes. These are the structure names from PDF
2190 1.7 (\texttt{StructTreeRoot} is a bit special, it is not really a structure
2191 name but nevertheless listed here):
2192 \ExplSyntaxOn %%
2193 \clist_clear:N\l_tmpa_clist
2194 \prop_map_inline:cn
2195 {g__tag_role_NS_pdf_prop}
2196 {
2197 \str_if_eq:eeT
2198 {#1}
2199 {\use_i:nn #2}
2200 { \clist_put_right:Nn \l_tmpa_clist {#1} }
2201 }
2202 \clist_use:Nn \l_tmpa_clist {,\c_space_tl }. %%
2203 \ExplSyntaxOff
2204
2205 \item The \emph{standard structure namespace for PDF 2.0}. The public
2206 name of the namespace is |tag/NS/pdf2|. This can be used to
2207 reference the namespace e.g. in attributes. These are more or less
2208 same types as in PDF. The following types have been removed from
2209 this set\footnote{They still can be used in a PDF 2.0 document!}:\\
2210 %
2211 \ExplSyntaxOn %
2212 \clist_clear:N\l_tmpa_clist
2213 \prop_map_inline:cn { g__tag_role_NS_pdf_prop }
2214 {
2215 \prop_if_in:cnF
2216 { g__tag_role_NS_pdf2_prop }
2217 {#1}
2218 { \clist_put_right:Nn \l_tmpa_clist {#1} }
2219 }
2220 \clist_use:Nn \l_tmpa_clist {,\c_space_tl },\\
2221 \ExplSyntaxOff %
2222 and the following are new:\\
2223 \ExplSyntaxOn %
2224 \clist_clear:N\l_tmpa_clist %
2225 \prop_map_inline:cn { g__tag_role_NS_pdf_prop }
2226 {
2227 \str_if_eq:eeF {#1}
2228 {\use_i:nn #2}
2229 { \clist_put_right:Nn \l_tmpa_clist {#1} }
2230 }
2231 \clist_use:Nn \l_tmpa_clist {,\c_space_tl }.
2232 \ExplSyntaxOff %
2233
2234 \item MathML 3.0 as an \emph{other namespaces}. The public name of
2235 the namespace is |tag/NS/mathml|. This can be used to reference the
2236 namespace e.g. in attributes. There are nearly 200 types in this
2237 name space, so I refrain from listing them here.
2238 \end{enumerate}
2239
2240 To allow to this more complicated setup the syntax of the
2241 \texttt{role/new-tag} key has been extended. It now takes as argument
2242 a key-value list with the following keys. A normal document shouldn't
2243 need the extended syntax, the simple syntax |section/H1| should in
2244 most cases do the right thing.
2245
2246 \begin{description}
2247 \item[\PrintKeyName{tag}] This is the name of the new type as it
2248 should then be used in \cs{tagstructbegin}.
2249
2250 \item[\PrintKeyName{tag-namespace}] This is the namespace of the new type.
2251 The value should be a shorthand of a namespace. The allowed values are
2252 currently |pdf|, |pdf2|, |mathml| and |user|. The default value (and
2253 recommended value for a new tag) is |user|. The public name of the user
2254 namespace is |tag/NS/user|. This can be used to reference the namespace
2255 e.g. in attributes.
2256
2257 \item[\PrintKeyName{role}] This is the type the tag should be mapped
2258 too. In a PDF 1.7 or earlier this is normally a type from the |pdf|
2259 set, in PDF 2.0 from the |pdf|, |pdf2| or |mathml| set. It can also
2260 be a user type, then this user tag must have been declared
2261 before. The PDF format allows mapping to be done transitively. But
2262 you should be aware that tagpdf can't (or more precisely won't)
2263 check if some unusual role mapping makes really sense, this lies in
2264 the responsibility of the author.
2265
2266 \item[\PrintKeyName{role-namespace}] The default value is the default
2267 namespace of the role: |pdf2| for all types in this set, |pdf| for
2268 the type which exist only in PDF 1.7, |mathml| for the MathML types,
2269 and for previously defined user types whatever namespace has been
2270 set there. With this key the value can be overwritten.
2271
2272 \item[unknown key] An unknown key is interpreted as a |tag/role|,
2273 this preserves the old syntax. So this two calls are equivalent:
2274
2275 \begin{lstlisting}
2276 \tagpdfsetup{role/new-tag = section/H1}
2277 \tagpdfsetup{role/new-tag = {tag=section,role=H1}}
2278 \end{lstlisting}
2279
2280 \end{description}
2281
2282 The exact effects of the keys depend on the PDF version. With PDF 1.7
2283 or older the namespace keys are ignored, with PDF 2.0 the namespace
2284 keys are use to setup the correct rolemaps. The |namespace| key is
2285 also used to define the default namespace if the type is used as a
2286 role or as tag in a structure.
2287
2288 \subsection{The \texttt{latex} namespace}
2289
2290 Starting with version 0.98 work has started to setup specific latex tags. In
2291 \PDF{} 2.0 in form of a special name space, with \PDF{} 1.7 or older the
2292 tags are role mapped. This is work in progress and bound to change.
2293
2294 \subsection{The \texttt{user} namespace}
2295
2296 As mentioned above new tags are created by default in the user namespace.
2297 Namespaces are identified through an ID which is normally given as an URI.
2298 By default tagpdf creates an random URI for the user namespace but it is possible to change that:
2299
2300 \begin{lstlisting}
2301 \tagpdfsetup{role/user-NS = somestring}
2302 \end{lstlisting}
2303
2304 will create use the URI \texttt{https://www.latex-project.org/ns/local/somestring} as ID for the namespace.
2305
2306
2307 \subsection{Fallback RoleMap}
2308
2309 As mentioned above PDF 2.0 support name spaces for tags. This is quite nice. At first because it
2310 avoid name clashes, but also because it allow to build a cleaner model of the document structure.
2311
2312 But sadly support for PDF 2.0 is still quite scarce and while most PDF readers have no problems
2313 to open and render a PDF 2.0 file they don't \enquote{see} the role mapping if name spaces are used.
2314 Therefore since version 0.98t \pkg{tagpdf} adds in PDF 2.0 files additionally also a global |/RoleMap|
2315 dictionary as a fallback for such processors.
2316
2317 \subsection{Mathml}
2318
2319 In PDF 2.0 mathml tags have their own name space and can be freely used.
2320 In PDF 1.7. they can only be used if they are rolemapped to a standard type.
2321 By default they are not added to the |/RoleMap| dictionary, but this can be
2322 forced with |\tagpdfsetup{role/mathml-tags}|. Please note that this adds mathml
2323 at the end of the document and overwrites tags with the same name without warning.
2324
2325
2326 \section{Checking parent-child rules}\label{sec:parent-child}
2327
2328 The \PDF{} references formulate various rules about whether a
2329 structure can be a child of another structure, e.g. a \texttt{Sect}
2330 can not be a child of \texttt{P}. In the \PDF{} 1.7 reference this
2331 rules were rather vague, in the \PDF{} 2.0 reference there is a quite
2332 specific matrix, which sadly misses some of the tags from \PDF{}
2333 1.7. The now released ISO norm 32005 addresses this problem and
2334 extends the matrix to cover tags from \PDF{} 1.7 and 2.0 (but it still
2335 misses the \texttt{math} tag and \texttt{mathml} tags).
2336
2337 The rules in the matrix are not a simple allowed/not allowed. Instead
2338 some rules determine that structure elements can appear only once in a
2339 parent, or that additional requirements can be found in the
2340 descriptions of the standard structure types, e.g. \texttt{Caption}
2341 often has to be the first element in the parent structure, and
2342 elements like \texttt{Part} and \texttt{Div} inherit restrictions from
2343 parent structures. External standards like \PDF/UA can add more
2344 rules.
2345
2346 Altogether this doesn't make it easy to check if a structure tree is
2347 conformant or not without slowing down the compilation a lot.
2348
2349 With version 0.98 some first steps to do checks (and to react to the
2350 result of a change) have been implemented. Some checks will led to
2351 warning directly, but the majority will only be visible if the
2352 log-level is increased.
2353
2354 Typical messages will look then like this
2355
2356 \begin{lstlisting}[mathescape]
2357 Package tagpdf Info: Checking Parent-Child 'pdf2:Document' --> 'latex:text-unit'
2358 Package tagpdf Info: Parent-Child 'Document' --> 'Part'.
2359 (tagpdf) Relation is 1 (='0..n') on line ...
2360
2361 Package tagpdf Warning: Parent-Child 'pdf2:Document' --> 'pdf2:Ruby'.
2362 (tagpdf) Relation is not allowed! on line ...
2363 (tagpdf) struct 67, text-unit --> struct 68, Ruby
2364 \end{lstlisting}
2365
2366
2367 The descriptions of the parent and child are rather verbose as the checks
2368 have to take role mapping and name spaces into account. The result of a
2369 check is a number---negative if the relation is not allowed, positive if
2370 allowed. The text in the parentheses show the symbols used in the
2371 \PDF-matrix.
2372
2373 Be aware
2374 \begin{itemize}
2375 \item This doesn't test all rules, it only implements (hopefully
2376 correctly) the matrix.
2377
2378 \item There can be differences between \PDF~1.7 and 2.0,
2379 e.g. \texttt{FENote} is role-mapped to \texttt{Note} in \PDF~1.7 and
2380 then has different containment rules.
2381
2382 \item The special tag \texttt{MC} stands for mc-chunks, so
2383 \enquote{real content} (the matrix has containments rules for this
2384 too).
2385
2386 \item Currently there is only only negative number \texttt{\textminus1}
2387 but that is bound to change, depending on if (and how) it is
2388 possible to \enquote{repair} a disallowed parent-child relation.
2389
2390 \item Warnings can be wrong.
2391
2392 \item The tags \texttt{Part}, \texttt{Div} and \texttt{NoStruct} are sometimes (not always) \enquote{transparent}, that means that it depends on their parent if a child tag is allowed. The code tries to
2393 detect this, but if a structure using one of these tags is stashed it can't identify the parent and so won't be able to check the rule
2394 \end{itemize}
2395
2396
2397 Parent-child checks are not fast, so there is an option to disable them
2398 \begin{lstlisting}
2399 \tagpdfsetup{debug/parent-child-check=off}
2400 \end{lstlisting}
2401 This will be speed up the compilation but should be used with care, parent-child checks are an important tool to detect faulty structures!
2402
2403 They can be reenabled with
2404 \begin{lstlisting}
2405 \tagpdfsetup{debug/parent-child-check=on}
2406 \end{lstlisting}
2407
2408 When using lualatex there is also the value \texttt{atend}. This disables the inline checks, but at the end of the document, the Lua code will go through the structures and check the rules.
2409
2410 There is also an online service \cite{showtags} which validates the structure with an Relax NG schema developed by the LaTeX Team, which can be found at \cite{showtags-script}.
2411
2412
2413 \section{\enquote{Real} space glyphs}\label{sec:spacechars}
2414
2415
2416 TeX uses only spaces (horizontal movements) to separate words. That
2417 means that a \PDF{} reader has to use some heuristic when copying text
2418 or reflowing the text to decide if a space is meant as a word boundary
2419 or e.g. as a kerning. Accessible document should use real space
2420 glyphs (U+0032) from a font in such places.
2421
2422 With the key \PrintKeyName{activate/spaces} you can activate such space
2423 glyphs.
2424
2425 With pdftex this will simply call the primitive
2426 \verb+\pdfinterwordspaceon+. pdftex will then insert at various places
2427 a char from a font called dummy-space. Attention! This means that at
2428 every space there are additional font switches in the \PDF{}: from the
2429 current font to the dummy-space font and back again. This will make
2430 the \PDF{} larger. As \verb+\pdfinterwordspaceon+ is a primitive
2431 function it can't be fine tuned or adapted. You can only turn it on
2432 and off and insert manually such a space glyph with
2433 \verb+\pdffakespace+.
2434
2435 With luatex (in luamode) |activate/spaces| is implemented with a
2436 Lua-function which is inserted in two callbacks and marks up the
2437 places where it seems sensible to inter a space glyph. Later in the
2438 process the space glyphs are injected -- the code will take the glyph
2439 from the current font if this has a space glyph or switch to the
2440 default latin modern font. The current code works reasonable well in
2441 normal text. |activate/spaces| can be used without actually tagging a
2442 document.
2443
2444 The key-value \PrintKeyName{debug/show=spaces} will show lines at the places where
2445 in Lua mode spaces are inserted and so can help you to find
2446 problematic places. For listings -- which have a quite specific
2447 handling of spaces -- you can find a suggestion in the example
2448 \texttt{ex-space-glyph-listings}.
2449
2450 \emph{Attention:} Even with real spaces copy\& pasting of code doesn't
2451 need to give the correct results: you get spaces but not necessarily
2452 the right number of spaces. The \PDF{} viewers I tried all copied four
2453 real space glyphs as one space. I only got the four spaces with the
2454 export to text or xml in the AdobePro.
2455
2456 \begin{docCommand}{pdffakespace}{}\end{docCommand}
2457
2458
2459 This is in pdftex a primitive. It inserts the dummy space glyph.
2460 \pkg{tagpdf} defines this command also for luatex -- attention if can
2461 perhaps insert break points.
2462
2463 \begin{docCommands}
2464 {
2465 {doc name=tag_spacechar_off:},
2466 {doc name=tag_spacechar_on:}
2467 }
2468 \end{docCommands}
2469
2470 The commands allow to switch on and off the insertion of space chars.
2471 With pdftex they map to primitive \cs{pdfinterwordspaceoff} and
2472 and \cs{pdfinterwordspaceon} which insert a whatsits and so act globally.
2473 The luatex implementation uses an attribute which is also set globally to stay
2474 more or less consistent with pdftex. In dvi-mode the commands do nothing.
2475
2476
2477 \section{Structure destinations}\label{sec:struct-dest}
2478
2479 Standard destinations (anchors for internal links) consist of a
2480 reference to a page in the pdf and instructions how to display
2481 it---typically they will put a specific coordinate in the left top
2482 corner of the viewer and so give the impression that a link jumped to
2483 the word in this place. But in reality they are not connected to the
2484 content.
2485
2486 Starting with pdf~2.0 destinations can in a tagged PDF also point to a
2487 structure (to a \texttt{/StructElem} object). GoTo links can then
2488 additionally to the \texttt{/D} key which points to a standard page
2489 destination also point to such a structure destination with an
2490 \texttt{/SD} key. Programs that e.g. convert such a PDF to html can then
2491 create better links. (According to the reference, PDF-viewer should prefer
2492 the structure destination over the page destination, but as far as it is
2493 known this isn't done yet.)
2494
2495 At first structure destinations (and GoTo links making use of it) could
2496 natively only be created with the dvipdfmx backend. With pdftex and
2497 lualatex it was only possible to create a restricted type which used only
2498 the \enquote{Fit} mode. Starting with \TeX{}live 2022 (earlier in miktex)
2499 both engines knew new keywords which allowed to create structure
2500 destination easily and support has been already added to the \PDF\
2501 management and \pkg{tagpdf}. In most cases it should simply work, but one
2502 should be aware that as one now has a destination that is actually tied to
2503 the content it gets more important to actually consider the context and
2504 the place where such destinations are created. It now makes a difference
2505 if the destination is created before the structure is opened or after so
2506 in some cases code that place destinations should be changed to place them
2507 inside the structure they belong too. One also has to consider the pages
2508 connected to the destinations: The structure destination is bound to the
2509 page where the structure \emph{begins}, if this differ from the page of
2510 the page destination (e.g. if the destination is created by a
2511 \verb+\phantomsection+ in the middle of a longer paragraph) then it may
2512 be necessary to surround destinations with a dummy structure (a Span or an
2513 Artifact) to get the right page number.
2514
2515 \section{Storing and reusing boxes}\label{sec:savebox}
2516
2517 \TeX{} allows to store material in boxes and to use these box once or
2518 multiple times in other places. This poses some challenges to tagging. The
2519 listings in the following examples uses low-level \TeX{} box commands to
2520 avoid that changes in the \LaTeX{} commands that improve tagging interfere
2521 in case you want to test this. To keep the examples short they don't show
2522 the needed \cs{ExplSyntaxOn}/\cs{ExplSyntaxOff}.
2523
2524 \subsection{Boxes without tagging commands}
2525
2526
2527 If no tagging commands were used (or if they were inactive) when the box
2528 was stored then there is no problem to use this box with
2529 pdf\LaTeX{}/generic mode in various places. So
2530
2531 \begin{lstlisting}
2532 \newbox\mybox
2533 The\setbox\mybox\hbox{yellow} duck
2534
2535 The \box\mybox{} sun
2536 \end{lstlisting}
2537
2538 will produce (assuming para tagging is activated) the paragraph structures
2539 \enquote{The duck} and \enquote{The yellow sun}.
2540
2541
2542 With Lua\LaTeX{}/Lua mode this is different: The nodes in the box will have
2543 the mc-attribute value attached which were active when the box was saved
2544 and this value is recorded as kid of the first paragraph. So when the Lua
2545 code later wanders through the box to find all kids of structure it will
2546 also find the content of the \cs{usebox}. This means with Lua\LaTeX{} we
2547 get the two paragraph structures \enquote{The duck yellow} and \enquote{The
2548 sun}.
2549
2550 The solution here is to reset the attributes before using the box:
2551
2552 \begin{lstlisting}
2553 The\setbox\mybox\hbox{yellow} duck
2554
2555 The \tag_mc_reset_box:N\mybox\box\mybox{} sun
2556 \end{lstlisting}
2557
2558
2559 The box can in both modes be used without problems many times.
2560
2561
2562 \subsection{Boxes with tagging commands}
2563
2564 We assume in the following that the box contains only well balanced tagging
2565 commands and no parts that are \enquote{untagged}. It should be possible to
2566 copy the whole box inside a \verb+\tagstructbegin+/\hspace{0pt}\verb+\tagstructend+
2567 pair. So the following is fine as box content
2568 \begin{lstlisting}
2569 box=\tagstructbegin{...}\tagmcbegin{} balanced content\tagmcend\tagstructend
2570 box=
2571 \tagmcbegin{}text\tagmcend
2572 \tagstructbegin{...}\tagmcbegin{} balanced content\tagmcend\tagstructend
2573 \tagmcbegin{}text\tagmcend
2574 \end{lstlisting}
2575 but this not (this case could probably be handled nevertheless with a bit
2576 care at least in Lua mode)
2577 \begin{lstlisting}
2578 box= text\tagmcend\tagstructbegin{...}...\tagstructend\tagmcbegin{}text
2579 \end{lstlisting}
2580 and this is absolutely unusable:
2581 \begin{lstlisting}
2582 box= text\tagmcend\tagstructbegin{...}\tagmcbegin{}text
2583 \end{lstlisting}
2584
2585 We also assume that we want to move the structure of the box to the place
2586 where the box is used (if the structure should stay where the box is saved,
2587 simply save it and that will happen). For this we must add a structure that
2588 we can stash and label.
2589
2590 \begin{lstlisting}
2591 \tag_mc_end_push: % interrupt an open mc
2592 \tagstructbegin{tag=NonStruct,stash}
2593 \edef\myboxnum{\tag_get:n{struct_num}} % store structure number
2594 \setbox\mybox\hbox %or \vbox or ...
2595 {content}
2596 \tagstructend
2597 \tag_mc_begin_pop:n{}% restart open mc
2598 \end{lstlisting}
2599
2600 At the place where the box is then used we also have to inject this
2601 structure:
2602
2603 \begin{lstlisting}
2604 \tag_mc_end_push: % interrupt an open mc
2605 \tag_struct_use_num:n {\myboxnum} % use structure
2606 \box\mybox % use box
2607 \tag_mc_begin_pop:n{}% restart open mc
2608 \end{lstlisting}
2609
2610 With pdf\LaTeX{} Boxes with tagging commands can currently be used only
2611 once. The tagging commands set labels and reusing the box gives multiple
2612 label warnings.
2613
2614 With Lua\LaTeX{} it is possible to reset the attributes as done with the
2615 untagged box and then to reuse at least the content.
2616
2617 \subsection{Detecting tagging commands}
2618
2619 It is possible to detect if a box contains tagging commands by comparing
2620 the state of the mc and structure counter:
2621 \begin{verbatim}
2622 \def\statebeforebox\inteval{\tag_get:n{struct_counter}+\tag_get:n{mc_counter}}
2623 \setbox\mybox ...
2624 %compare numbers against \statebeforebox
2625 \end{verbatim}
2626
2627
2628
2629 \subsection{Putting everything together}
2630
2631 To tag boxes that can be both (without tagging commands or with balanced
2632 tagging commands) the following strategy can be used:
2633
2634 \begin{itemize}
2635 \item when storing the box put around it a structure as needed by the
2636 tagged variant:
2637 \begin{verbatim}
2638 \tag_mc_end_push: % interrupt an open mc
2639 \tagstructbegin{tag=NonStruct,stash}
2640 \edef\myboxnum{\tag_get:n{struct_num}} % store structure number
2641 \def\statebeforebox{\inteval{\tag_get:n{struct_counter}+\tag_get:n{mc_counter}}}
2642 \setbox\mybox\hbox %or \vbox or ...
2643 {content}
2644 %check if there is tagging content and store that
2645 \tagstructend
2646 \tag_mc_begin_pop:n{}% restart open mc
2647 \end{verbatim}
2648
2649 \item when using the box the first time
2650 \begin{itemize}
2651 \item if it has no tagging commands then reset the attribute and use
2652 the box.
2653 \begin{verbatim}
2654 \ExplSyntaxOn
2655 \let\tagmcresetbox\tag_mc_reset_box:N %make a copy
2656 \ExplSyntaxOff
2657 ...
2658 The \tagmcresetbox\mybox\box\mybox{} sun
2659 \end{verbatim}
2660 The stashed \texttt{NonStruct} structure is then thrown away.
2661 \item if there is a structure then use the stashed structure
2662 \begin{verbatim}
2663 \tag_mc_end_push: % interrupt an open mc
2664 \tag_struct_use_num:n {\myboxnum} % use structure
2665 \box\mybox % use box
2666 \tag_mc_begin_pop:n{}% restart open mc
2667 \end{verbatim}
2668 \end{itemize}
2669
2670 \item if the box is used a second time then throw an error with
2671 pdf\LaTeX{}. With Lua\LaTeX{} reset the attributes and issue a
2672 warning.
2673
2674 \end{itemize}
2675
2676
2677 \section{Accessibility is not only tagging}
2678
2679
2680 A tagged \PDF{} is needed for accessibility but this is not enough. As
2681 already mentioned there are more requirements:
2682
2683 \begin{itemize}
2684 \item The language must be declared by adding a \texttt{/Lang xx-XX} to
2685 the \PDF{} catalog or -- if the language changes for a part of the
2686 text to the structure or the marked content. Setting the document
2687 language can be done with the \texttt{lang} option of
2688 \cs{DocumentMetadata}. For settings in marked content and structure
2689 the \texttt{lang} key can be used too.
2690
2691 \item All characters must have a Unicode representation or a suitable
2692 alternative text. With lualatex and open type (Unicode) fonts this
2693 is normally not a problem. With pdflatex it could need additional
2694 \verb+\pdfglyphtounicode+ commands.
2695
2696 \item Hard and soft hyphen must be distinct. In luamode this is now handled through
2697 the \texttt{activate/softhyphen} key. For pdftex no solution is known.
2698
2699 \item Spaces between words should be space glyphs and not only a
2700 horizontal movement. See section~\ref{sec:spacechars}.
2701
2702 \item Various small infos must be present in the catalog dictionary,
2703 info dictionary and the page dictionaries, e.g. metadata like title.
2704 This can be done with the options of \cs{DocumentMetadata}. See the
2705 documentation of \texttt{l3pdfmeta} for details.
2706 \end{itemize}
2707
2708 \section{CSS style definition for derivation to html}
2709
2710 Derivation to html (\cite{derivation}
2711 implemented by, e.g., ngpdf) can be improved by attaching CSS style definitions
2712 in associated files with relationship supplement to the StructTreeRoot.
2713
2714 Such CSS style definitions can be given in two ways:
2715 \begin{itemize}
2716 \item In files with the extension \texttt{.css}. Such files should contain only
2717 CSS style definitions. A derivation processor like ngpdf will store these files and include them with an
2718 \verb+<link rel=stylesheet href=...>+ in the head of the html.
2719 \item In files with the extension \texttt{.html}. Such files should contain CSS style
2720 definitions inside one (or more) \verb+<style>...</style>+ html tags. The content of these
2721 files are copied directly into the head of the derived html.
2722 \end{itemize}
2723
2724 By default (if tagging is active) tagpdf embeds now such CSS style definitions.
2725 Currently the list of files is rather short and consists of two files (with extension
2726 .html and \verb+<style>...</style>+ html tags) which are provided by the tagpdf package:
2727 \begin{itemize}
2728 \item \texttt{latex-align-css.html} which improves the styling of amsmath alignments tagged with MathML.
2729 \item \texttt{latex-list-css.html} which improves the style of list environments.
2730 \end{itemize}
2731
2732 A number of configuration keys for \cs{tagpdfsetup} are provided
2733 \begin{description}
2734 \item[\PrintKeyName{attach-css}] Boolean, initially true.
2735 It is possible to suppress the embedding of these files by setting this key
2736 to \texttt{false}, \texttt{attach-css=true} or \texttt{attach-css} reverts this again.
2737 \item[\PrintKeyName{css-list}] With this key developers can overwrite the list, e.g. with \verb+css-list={file1,file2}+ (where all files should have either the extension \texttt{html} or \texttt{css}. \verb+css-list=+ clears
2738 the list (and so suppresses the embedding too).
2739 \item[\PrintKeyName{css-list-remove}] This key allows to remove a file from
2740 the list, e.g. \verb+css-list-remove=latex-list-css.html+.
2741 \item[\PrintKeyName{css-list-add}] this key allows to add more files to the
2742 list, e.g., \verb+css-list-add=my-fancy-align-css.html+.
2743 It is also possible to attach a .css-file in this way.
2744 \end{description}
2745
2746 These keys do not affect files added directly with \PrintKeyName{root-supplemental-file}.
2747
2748 The files in this list are attached at the end of the compilation (and so normally after the files attached with root-supplemental-file)
2749 but you shouldn't rely on this or on a specific order of the embedding in the html.
2750
2751
2752 \section{Debugging}
2753
2754 While developing commands and tagging a document, it can be useful to get
2755 some info about the current structure. For this a show command is provided
2756
2757 \begin{docCommand}{ShowTagging}{\marg{key-val}}\end{docCommand}
2758
2759 This command takes as argument a key-val list which implements a number of show options.
2760
2761 \begin{description}
2762 \item[\PrintKeyName{mc-data}] This key is relevant for luamode only.
2763 It shows the data of all mc-chunks created so far.
2764 It is accurate only after shipout, so typically should be issued after a
2765 newpage. The value is a positive integer and sets the first mc-shown. If no value
2766 is given, 1 is used and so all mc-chunks created so far are shown.
2767
2768 \item[\PrintKeyName{mc-current}] This key shows the number and the tag of
2769 the currently open mc-chunk. If no chunk is open it shows only the
2770 state of the absolute counter. It works in all mode,
2771 but the output in luamode looks different.
2772
2773 \item[\PrintKeyName{struct-stack}] This key shows the current
2774 structure stack. Typically it will contain at
2775 least |root| and |Document|.
2776 With the value |log| the info is only
2777 written to the log-file, |show| stops the compilation and shows on the terminal.
2778 If no value is used, then the default is |show|.
2779
2780 \item[\PrintKeyName{debug/structures}] This key is only available if
2781 the package \pkg{tagpdf-debug} has been loaded too. It takes as
2782 value a number (the default is 0), and shows on the terminal and
2783 in the log information about all structures with a number equal or
2784 larger than the number. The data avoids to show PDF object numbers
2785 to make it more usable for test suites.
2786 \end{description}
2787
2788
2789
2790 \section{To-do}
2791
2792
2793 \begin{itemize}
2794 \item Add commands and keys to enable/disable the checks.
2795 \item Check/extend the code for language tags.
2796 \item Think about math (progress: examples using luamml, associated files exists).
2797 \item Think about Links/Annotations
2798 (progress: mostly done, see section~\ref{sec:link+annot} and the code in \pkg{l3pdffield})
2799 \item Keys for alternative and actualtext. How to define the input encoding? Like in Accsupp?
2800 (progress: keys are there, but encoding interface needs perhaps improving)
2801 \item Check twocolumn documents
2802 \item Examples
2803 \item Write more Tests
2804 \item Write more Tests
2805 \item Unicode
2806 \item Hyphenation char
2807 \item Think about included (tagged) \PDF{}. Can one handle them?
2808 \item Improve the documentation (progress: it gets better)
2809 \item Tag as proof of concept the documentation (nearly done)
2810 \item Document the code better (progress: mostly done)
2811 \item Create dtx (progress: done)
2812 \item Find someone to check and improve the Lua code
2813 \item Move more things to Lua in the luamode
2814 \item Find someone to check and improve the rest of the code
2815 \item Check differences between \PDF{} versions 1.7 and 2.0. (progress:
2816 WIP, namespaces done)
2817 \item bidi?
2818 \end{itemize}
2819
2820
2821 \makeatletter % fix TOC of History
2822 \addtocontents{toc}{\def\string\l@subsection{\string\@dottedtocline{2}{1.5em}{3em}}}
2823 \makeatother
2824
2825
2826 \section{History}
2827
2828 This section lists important changes during the development of the package.
2829 More can be found in the \texttt{CHANGELOG.MD} and by checking the git
2830 commits.
2831
2832 \subsection{Changes in 0.3}
2833
2834 In this version I improved the handling of alternative and actual text. See
2835 section~\ref{sec:alt}. This change meant that the package relies on the
2836 module \texttt{l3str-convert}.
2837
2838 I no longer try to (pdf-)escape the tag names: it is a bit unclear how to
2839 do it at best with luatex. This will perhaps later change again.
2840
2841 \subsection{Changes in 0.5}
2842
2843 I added code to handle attributes and attribute classes, see
2844 section~\ref{sec:attributes} and corrected a small number of code errors.
2845
2846 I added code to add \enquote{real} space glyphs to the \PDF{}, see section
2847 \ref{sec:spacechars}.
2848
2849 \subsection{Changes in 0.6}
2850
2851 \textbf{Breaking change!} The attributes used in luamode to mark the
2852 MC-chunks are no longer set globally. I thought that global attributes
2853 would make it easier to tag, but it only leads to problem when e.g. header
2854 and footer are inserted. So from this version on the attributes are set
2855 locally and the effect of a \verb+\tagmcbegin+ ends with the current group.
2856 This means that in some cases more \verb+\tagmcbegin+ are needed and this
2857 affected some of the examples, e.g. the patching commands for sections with
2858 KOMA. On the other side it means that quite often one can omit the
2859 \verb+\tagmcend+ command.
2860
2861
2862 \subsection{Changes in version 0.61}
2863
2864 \begin{itemize}
2865 \item internal code adaptions to expl3 changes.
2866 \item dropped the compresslevel key -- probably not needed.
2867 \end{itemize}
2868
2869
2870 \subsection{Changes in version 0.8}
2871
2872 \begin{itemize}
2873 \item As a first step to include the code proper in the \LaTeX\ kernel
2874 the module name has changed from \texttt{uftag} to \texttt{tag}. The
2875 commands starting with |\uftag| will stay valid for some time but
2876 then be deprecated.
2877
2878 \item \textbf{Breaking change!} The argument of \texttt{role/new-attribute} (old key name: \texttt{newattribute})
2879 option should no longer add the dictionary bracket \verb+<<..>>+,
2880 they are added by the code.
2881
2882
2883 \item \textbf{Breaking change!} The package now requires the new PDF
2884 management as provided for now by the package
2885 \pkg{pdfmanagement-testphase}. \pkg{pdfmanagement-testphase},
2886 prepares the ground for better support for tagged PDF in \LaTeX{}. It
2887 is part of a larger project to automatically generate tagged PDF
2888 \url{https://www.latex-project.org/news/2020/11/30/tagged-pdf-FS-study/}
2889
2890 \item Support to add associated files to structures has been added with
2891 new keys \texttt{AF}, \texttt{AFinline} and \texttt{AFinline-o}.
2892
2893 \item \textbf{Breaking change!} The support for other 8-bit input
2894 encodings has been removed. utf8 is now the required encoding.
2895
2896 \item The keys |lang|, |ref| and |E| have been added for structures.
2897
2898 \item The new hooks of \LaTeX\ are used to tagged many paragraphs
2899 automatically. The small red numbers around paragraphs in the
2900 documentation show them in action. The main problem here is not to
2901 tag a paragraph, but to avoid to tag too many: paragraphs pop up in
2902 many places.
2903 \end{itemize}
2904
2905 \subsection{Changes in version 0.81}
2906
2907 \begin{itemize}
2908 \item Hook code to tag links (URI and GoTo type) have been added. So
2909 normally they should simply work if tagging is activated.
2910
2911 \item Commands and keys to allow automatic paragraph tagging have been
2912 added. See section~\ref{sec:paratagging}. As can be seen in this
2913 documentation the code works quite good already, but one should be
2914 aware that \enquote{paragraphs} can appear in many places and
2915 sometimes there are even more paragraph begin than ends.
2916
2917 \item A key to test if local or global setting of the mc-attributes in
2918 luamode is more sensible, see \ref{sec:global-local} for more
2919 details.
2920
2921 \item New commands to store and reset mc-tags.
2922
2923 \item PDF 2.0 namespaces are now supported.
2924 \end{itemize}
2925
2926 \subsection{Changes in version 0.82}
2927
2928 A command |\tag_if_active:TF| to test if tagging is active has been added.
2929 This allow external packages to write conditional code.
2930
2931 The commands |\tag_struct_parent_int:| and |\tag_struct_insert_annot:nn|
2932 have been added. They allow to add annotations to the structure.
2933
2934
2935 \subsection{Changes in version 0.83}
2936
2937 |\tag_finish_structure:| has been removed, it is no longer a public
2938 command.
2939
2940 \subsection{Changes in version 0.90}
2941
2942 \begin{itemize}
2943 \item Code has been cleaned up and better documented.
2944
2945 \item \textbf{More engines supported} The generic mode of \pkg{tagpdf}
2946 now works (theoretically, it is not much tested) with all engines
2947 supported by the \PDF\ management. So compilations with Xe\LaTeX{} or
2948 with dvips should work. But it should be noted that these engines and
2949 backends don't support the |interspaceword| option. With Xe\LaTeX{}
2950 it is perhaps possible implement something with
2951 |\XeTeXinterchartoks|, but for the dvips route I don't see an option
2952 (apart from lots of manual macros everywhere).
2953 \item \textbf{MC-attributes are global again} In\sidenote{Breaking
2954 change!} version 0.6 the attributes used in luamode to mark the
2955 MC-chunks were no longer set globally. This avoided a number of
2956 problems with header and footer and background material, but further
2957 tests showed that it makes it difficult to correctly mark things like
2958 links which have to interrupt the current marking code---the
2959 attributes couldn't easily escape groups added by users. See
2960 section~\ref{sec:global-local} for more details.
2961 \item \textbf{key global-mc removed:} Due to the changes in the attribute
2962 keys this key is not longer needed.
2963 \item \textbf{key check-tags removed:} It doesn't fit. Checks are handled
2964 over the logging level.
2965 \item |\tagpdfget| has been removed, use the expl3 version if needed.
2966 \item The show commands |\showtagpdfmcdata|, |\showtagpdfattributes|,
2967 |\showtagstack| have been removed and replaced by a more flexible
2968 command |\ShowTagging|.
2969 \item The commands |\tagmcbegin| and |\tagmcend| no longer ignore
2970 following spaces or remove earlier one. While this is nice in some
2971 places, it also ate spaces in places where this wasn't expected. From
2972 now on both commands behave exactly like the expl3 versions.
2973 \item The Lua-code to add real space glyphs has been separated from the
2974 tagging code. This means that |activate/spaces| now works also if
2975 tagging is not active.
2976 \item The key |activate| has been added, it open the first structure, see
2977 above.
2978 \end{itemize}
2979
2980 \subsection{Changes in version 0.92}
2981
2982 \begin{itemize}
2983 \item support for page breaks in pdftex has been added, see
2984 section~\ref{sec:splitpara},
2985
2986
2987 \item header and footer are tagged as artifacts automatically, see
2988 section~\ref{sec:header-footer}.
2989
2990 \item keys \texttt{alttext-o} and \texttt{actualtext-o} has been removed.
2991 \texttt{alttext} and \texttt{actualtext} will now expand once.
2992
2993 \end{itemize}
2994
2995 \subsection{Changes in version 0.93}
2996
2997 \begin{itemize}
2998 \item Support for associated files in the root element (key
2999 \texttt{root-AF}) has been added. This allow e.g. to add a css-file
3000 which is be used if the \PDF\ is converted to html.
3001
3002 \item First steps have been done to adapt the package to planned changes
3003 in \LaTeX{}: The command \cs{DocumentMetadata} will be added to the
3004 format and will take over the role of \cs{DeclareDocumentMetadata}
3005 from \pkg{pdfmanagement-testphase} and additionally will also load
3006 the pdf management code. This will simplify the documents as it will
3007 no longer be needed to load the package.
3008
3009 \item The package has now support for \enquote{structure destinations}.
3010 This is a new type of destinations in \PDF~2.0. For pdftex and luatex
3011 this requires new binaries. They will be included in texlive 2022,
3012 miktex already has the new pdftex, the new luatex will probably
3013 follow soon.
3014
3015 \item The commands \cs{tagpdfifluatexT}, \cs{tagpdfifluatexTF} has been
3016 removed \cs{tagpdfifpdftexT},
3017
3018 \end{itemize}
3019
3020 \subsection{Changes in version 0.94}
3021
3022 In this version a small package, \pkg{tagpdf-base} has been added. It
3023 provides no-op versions of the main expl3 user commands for packages that
3024 want to support tagging but can't be sure if the \pkg{tagpdf} package has
3025 been loaded.
3026
3027 \subsection{Changes in version 0.95}
3028
3029 Small bug fixes.
3030
3031 \subsection{Changes in version 0.96}
3032
3033 \begin{itemize}
3034 \item The \texttt{alttext} key has been renamed to \texttt{alt}, the
3035 other key name exists as alias.
3036
3037 \item The new command |\tag_struct_object_ref:n| allows to create the
3038 object reference of a structure.
3039
3040 \item a new key \texttt{parent} has been added to allow structures to
3041 choose their parent structure.
3042
3043 \item a new option \texttt{paratag} allows to change the tag name used
3044 for the automatically tagged paragraphs.
3045
3046 \item the commands |\tag_start:|, |\tag_stop:|, |\tag_stop:n| and
3047 |\tag_start:n| allow to stop and start tagging (for example in trial
3048 typesetting).
3049
3050 \item Small bug fixes.
3051 \end{itemize}
3052
3053 \subsection{Changes in version 0.98}
3054
3055 \begin{itemize}
3056 \item The declarations of tag namespaces have been externalized and are
3057 now read from files when \pkg{tagpdf} is loaded.
3058
3059 \item The \PDF{} format (and some of the standards) declare various
3060 parent-child rules for structure tags. A first step to implement this
3061 rules and check if they are fulfilled have been done. More
3062 information can be found in section~\ref{sec:parent-child}.
3063
3064 \item As a side effect of the new rule checking, the requirements for new
3065 tags have been tightened: Adding a new tag with add-new-tag now
3066 requires that the target role is defined. Unknown roles error.
3067
3068 \item |\tagmcbegin| no longer requires that a tag is set, instead if will
3069 pick up the tag name from the surrounding structure.
3070
3071 \item Structure destination are now created also with \PDF
3072 \textless\,2.0. They shouldn't harm and can improve the html export.
3073
3074 \end{itemize}
3075
3076 \subsection{Changes in version 0.98a}
3077 Small bug fixes in code and documentation.
3078
3079 \subsection{Changes in version 0.98b}
3080 The main change is from now on every structure has an ID and an IDtree is
3081 added. The ID of a structure can be retrieved with |\tag_get:n|
3082 see~\ref{sec:retrieve}.
3083
3084
3085 \subsection{Changes in version 0.98e}
3086
3087 \begin{itemize}
3088 \item The main change is that the automatic paratagging uses now a
3089 two-level structure. This accompanies development in the \LaTeX\ github
3090 in the \texttt{latex-lab} package regarding the tagging of blocks like
3091 lists or verbatim. See~\ref{sec:paratagging} and also
3092 \texttt{latex-lab-block-tagging.dtx} for more background.
3093
3094 \item The command |tag_struct_end:n| has been add to improve debugging.
3095 \end{itemize}
3096
3097 \subsection{Changes in version 0.98k}
3098
3099 The luamode has been adapted and now allows also the compilation with
3100 dvilualatex. By default it will insert specials for \texttt{dvips} into the
3101 dvi. But be aware that \texttt{dvips} can normally not be used as it can't
3102 handle open type fonts, and extended version would be needed which isn't in
3103 texlive yet. It is also possible to use \texttt{dvipdfmx} as backend (which
3104 already has support for open type fonts), for this you need to use
3105 \texttt{backend=dvipdfmx} in the \cs{DocumentMetadata} command. Real space
3106 chars will work, but are currently not taken from the current font. This
3107 will be improved in the next luaotfload version. The compilation with
3108 dvilualatex is not much tested yet.
3109
3110 \subsection{Changes in version 0.98l}
3111
3112 In 2023 the primitives to write literal code into the pdf have been
3113 extended in all engines and now allow to delay the expansion of their
3114 argument to the shipout. This made it possible to greatly simplify and
3115 speed up the code used in generic mode to number the MC-chunks. In most
3116 cases building the structure should now need only two or three
3117 compilations. The new code requires a current pdfmanagement-testphase and
3118 is then used automatically if the new engines are detected.
3119
3120 \subsection{Changes in version 0.99f}
3121
3122 Deprecated |\tag_start:|, |\tag_stop:|, |\tag_stop:n| and
3123 |\tag_start:n| in favor of |\tag_suspend:n| and |\tag_resume:n|.
3124
3125
3126 \subsection{Changes in version 0.99m}
3127
3128 Removed/deactivated package options |luamode| and |genericmode|.
3129 Added first support for pronunciation hints, the |phoneme| key.
3130
3131 \printbibliography[heading=bibintoc]
3132
3133
3134
3135 \appendix
3136
3137
3138
3139 \section{Some remarks about the \PDF{} syntax}
3140
3141
3142 This is not meant as a full reference only as a background to make the
3143 examples and remarks easier to understand.
3144
3145
3146 \begin{description}
3147 \item[postfix notation] \PDF{} uses in various places postfix
3148 notation. This means that the operator is behind its arguments:
3149
3150 \begin{tikzpicture}[baseline=(c.base),alt={Illustration of postfix notation}]
3151 \node[arg](a1) {18};
3152 \node[arg,right=of a1.east](a2) {0};
3153 \node[operator,right= of a2.east](c) {obj};
3154 \draw[->] (c.south) --++(0,-2mm) -| (a1);
3155 \draw[->] (c.south) --++(0,-2mm) -| (a2);
3156 \end{tikzpicture}
3157
3158 \begin{tikzpicture}[baseline=(c.base),alt={Illustration of postfix notation}]
3159 \node[arg](a1) {18};
3160 \node[arg,right=of a1.east](a2) {0};
3161 \node[operator,right= of a2.east](c) {R};
3162 \draw[->] (c.south) --++(0,-2mm) -| (a1);
3163 \draw[->] (c.south) --++(0,-2mm) -| (a2);
3164 \end{tikzpicture} (a reference (operator R) to an object
3165
3166
3167 \begin{tikzpicture}[baseline=(c.base),alt={Illustration of postfix notation}]
3168 \node[arg](a1) {1};
3169 \node[arg,right = of a1.east](a2) {0};
3170 \node[arg,right = of a2.east](a3) {0};
3171 \node[arg,right = of a3.east](a4) {1};
3172 \node[arg,right = of a4.east](a5) {100.2};
3173 \node[arg,right = of a5.east](a6) {742};
3174 \node[operator,right = of a6.east](c) {Tm};
3175 \draw[->] (c.south) --++(0,-2mm) -| (a6);
3176 \draw[->] (c.south) --++(0,-2mm) -| (a5);
3177 \draw[->] (c.south) --++(0,-2mm) -|(a4);
3178 \draw[->] (c.south) --++(0,-2mm) -|(a3);
3179 \draw[->] (c.south) --++(0,-2mm) -| (a2);
3180 \draw[->] (c.south) --++(0,-2mm) -|(a1);
3181 \end{tikzpicture}
3182
3183 \begin{tikzpicture}[baseline=(c.base),alt={Illustration of postfix notation}]
3184 \node[arg](a1) {/P};
3185 \node[arg,right = of a1.east](a2) {<</MCID 0>>};
3186 \node[operator,right = of a2.east](c) {BDC};
3187 \draw[->] (c.south) --++(0,-2mm) -| (a1);
3188 \draw[->] (c.south) --++(0,-2mm) -| (a2);
3189 \end{tikzpicture}
3190
3191
3192 \item[Names] \PDF{} knows a sort of variable called a
3193 \enquote{name}. Names start with a slash and may include any regular
3194 characters, but not delimiter or white-space characters. Uppercase
3195 and lowercase letters are considered distinct: \texttt{/A} and
3196 \texttt{/a} are different names. \verb+/.notdef+ and
3197 \verb+/Adobe#20Green+ are valid names.
3198
3199 Quite a number of the options of \texttt{tagpdf} actually define
3200 such a name which is later added to the \PDF{}. I recommend
3201 \emph{strongly} not to use spaces and exotic chars in such
3202 names. While it is possible to escape such names it is rather a pain
3203 when moving them through the various lists and commands and quite
3204 probably I forgot some place where it is needed.
3205
3206 \item[Strings]There are two types of strings: \emph{Literal strings}
3207 are enclosed in round parentheses. They normally contain a mix of
3208 ascii chars and octal numbers:
3209
3210
3211 \verb+(gr\374\377ehello[]\050\051)+.
3212
3213
3214 \emph{Hexadezimal strings} are enclosed in angle brackets. They
3215 allow for a representation of all characters the whole Unicode
3216 ranges. This is the default output of lualatex.
3217
3218
3219 \texttt{<003B00600243013D0032>}.
3220
3221
3222
3223
3224 \item[Arrays] Arrays are enclosed by square brackets. They can contain
3225 all sort of objects including more arrays. As an example here an
3226 array which contains five objects: a number, an object reference, a
3227 string, a dictionary and another array. Be aware that despite the
3228 spaces \texttt{15 0 R} is \emph{one} element of the array.
3229
3230
3231 \mbox{\texttt{[0 15 0 R (hello) <</Type /X>> [1 2 3]]}}
3232
3233
3234 \begin{tikzpicture}[baseline=(a1.base),alt={Illustration of array}]
3235 \node[arg](a1) {0};
3236 \node[arg,right = of a1.east](a2) {15 0 R};
3237 \node[arg,right = of a2.east](a3) {(hello)};
3238 \node[arg,right = of a3.east](a4) {<</Type /X>>};
3239 \node[arg,right = of a4.east](a5) {[1 2 3]};
3240 \end{tikzpicture}
3241
3242
3243 \item[Dictionaries] Dictionaries are enclosed by double angle
3244 brackets. They contain key-value pairs. The key is always a
3245 name. The value can be all sort of objects including more
3246 dictionaries. It doesn't matter in which order the keys are given.
3247
3248 Dictionaries can be written all in one line:\\
3249 \texttt{<</Type/Page/Contents 3 0 R/Resources 1 0 R/Parent 5 0 R>>}\\
3250 but at least for examples a layout with line breaks and indentation is more readable:
3251
3252 \begin{lstlisting}
3253 <<
3254 /Type /Page
3255 /Contents 3 0 R
3256 /Resources 1 0 R
3257 /MediaBox [0 0 595.276 841.89]
3258 /Parent 5 0 R
3259 >>
3260 \end{lstlisting}
3261
3262
3263
3264
3265 \item[(indirect) objects] These are enclosed by the keywords
3266 \texttt{obj} (which has two numbers as prefix arguments) and
3267 \texttt{endobj}. The first argument is the object number, the
3268 second a generation number -- if a \PDF{} is edited objects with a
3269 larger generation number can be added. As with pdflatex/lualatex
3270 the \PDF{} is always new we can safely assume that the number is
3271 always 0. Objects can be referenced in other places with the
3272 \texttt{R} operator. The content of an object can be all sort of
3273 things.
3274
3275 \item[streams] A stream is a sequence of bytes. It can be long and is
3276 used for the real content of \PDF{}: text, fonts, content of
3277 graphics. A stream starts with a dictionary which at least sets the
3278 \texttt{/Length} name to the length of the stream followed by the
3279 stream content enclosed by the keywords \texttt{stream} and
3280 \texttt{endstream}.
3281
3282 Here an example of a stream, an object definition and reference. In the
3283 object 2 (a page object) the \texttt{/Contents} key references the
3284 object 3 and this then contains the text of the page in a stream.
3285 \texttt{Tf}, \texttt{Tm} and \texttt{TJ} are (postfix) operators, the
3286 first chooses the font with the name \texttt{/F15} at the size 10.9,
3287 the second displaces the reference point on the page and the third
3288 inserts the text.
3289
3290 \begin{lstlisting}
3291 % a page object (shortened)
3292 2 0 obj
3293 <<
3294 /Type/Page
3295 /Contents 3 0 R
3296 /Resources 1 0 R
3297 ...
3298 >>
3299 endobj
3300
3301 %the /Contents object (/Length value is wrong)
3302 3 0 obj
3303 <</Length 153 >>
3304 stream
3305 BT
3306 /F15 10.9 Tf 1 0 0 1 100.2 746.742 Tm [(hello)]TJ
3307 ET
3308 endstream
3309 endobj
3310 \end{lstlisting}
3311
3312 In such a stream the \texttt{BT}--\texttt{ET} pair encloses texts while
3313 drawing and graphics are outside of such pairs.
3314
3315 \item[Number tree] This is a more complex data structure that is meant to
3316 index objects by numbers. In the core is an array with number-value
3317 pairs. A simple version of number tree which has the keys 0 and 3 is
3318
3319 \begin{lstlisting}
3320 6 0 obj
3321 <<
3322 /Nums [
3323 0 [ 20 0 R 22 0 R]
3324 3 21 0 R
3325 ]
3326 >>
3327 endobj
3328 \end{lstlisting}
3329
3330 This maps 0 to an array and 2 to the object reference \texttt{21 0 R}.
3331 Number trees can be split over various nodes -- root, intermediate and
3332 leaf nodes. We will need such a tree for the \emph{parent tree}.
3333
3334 \end{description}
3335
3336
3337 \end{document}
3338
3339