Debian Codegraph @

texmf-dist/doc/latex/newpax/newpax.tex

raw
1 % !Mode:: "TeX:UTF-8:Main"
2 \makeatletter
3 \def\UlrikeFischer@package@version{0.58}
4 \def\UlrikeFischer@package@date{2026-05-09}
5 \makeatother
6 \RequirePackage{pdfmanagement}
7 \SetKeys[document/metadata]{pdfversion=1.7,lang=en-UK}
8
9 \documentclass[DIV=12,parskip=half-,bibliography=totoc]{scrartcl}
10 \usepackage{scrlayer-scrpage}
11 \usepackage{fontspec}
12 \setmainfont{Heuristica}
13 \usepackage{unicode-math}
14 \usepackage[english]{babel}
15 \usepackage[autostyle]{csquotes}
16 \usepackage{microtype}
17 \usepackage{listings}
18 \def\lstlanguagefiles
19 {lstlang0.sty,lstlang1.sty,lstlang2.sty,lstlang3.sty}
20
21 \lstset{basicstyle=\ttfamily, columns=fullflexible,language=[LaTeX]TeX,
22 escapechar=*,
23 commentstyle=\color{green!50!black}\bfseries}
24 \usepackage[pdfdisplaydoctitle=true,hyperfootnotes=false]{hyperref}
25 \hypersetup{pdftitle=The newpax package,pdfauthor=Ulrike Fischer}
26 \usepackage{ydoc-desc}
27
28 \usepackage{newpax}
29 \directlua{require("newpax")}
30 % write pax and .newpax files
31 \directlua
32 {
33 newpax.writenewpax("doc-input1")
34 newpax.writenewpax("doc-input2")
35 newpax.writepax("doc-input1")
36 newpax.writepax("doc-input2")
37 }
38 \newpaxsetup{usefileattributes}
39
40 \title{The \pkg{newpax} package, v\csname UlrikeFischer@package@version\endcsname}
41 \subtitle{Reinserting annotations from included pdf file}
42 \date{\csname UlrikeFischer@package@date\endcsname}
43 \author{Ulrike Fischer\thanks{[email protected]}}
44
45 \begin{document}
46 \maketitle
47
48 \section{Introduction}
49
50 Links in a PDF are created with annotation objects. Such an object is not connected to the content
51 or text, but simply describes an (rectangular) area on the page and defines an action if the
52 cursor is in the area. The coordinates of the area are given in absolute page coordinates.
53 The action of such an annotation can be an external URL, but also an internal destination.
54 Such destination are objects describing a page and some instructions how to display the page---again using absolute coordinates.
55
56 When a PDF is included in another PDF---may it be with \cs{includegraphics} or with \cs{includepdf}\hspace{0pt}---\hspace{0pt}the annotation coordinates no longer make sense as they don't refer to the receiving page (and often the action of an annotation doesn't make sense either), so all TeX-engines and backends strip them away when including a PDF: the net effect is that external and internal links are lost.
57
58 The \pkg{pax} package from Heiko Oberdiek offers a solution for this problem: it extracts all the annotations
59 and destinations of the included PDF in a text file, does some clever recalculations of their coordinates and reinserts them.
60 The package works basically fine but has a few drawbacks: To collect the annotation one has to run an external
61 java program which relies on a now outdated library, and it works only with pdf\LaTeX{}.
62
63 The \pkg{newpax} tries to address these problems. It offers a lua script to extract the annotations.
64 The script can be used with lua(la)tex and no external tools are needed. The annotations can
65 then be reinserted either with the \pkg{pax.sty} or with the new \pkg{newpax.sty} whose code is based in large parts on the \pkg{pax} package: it uses its data structure and the original code to calculate the coordinates (with a few minor bug corrections), but the pdf\LaTeX{} primitives have been replaced by commands from the \LaTeX{} PDF management in \pkg{pdfmanagement-testphase} so it should works with all major engines and backends (with the exception of dvips).
66
67
68 \section{Quick use instructions}
69
70 \subsection{Step 1: extract and collect the annotations}
71
72 The lua script offers a function which take as argument the name of a PDF (without the extension).
73 The function can be used in some lua scripts but also in a document which then must be compiled
74 with lualatex.
75
76 \lstinputlisting[caption=doc-extract-newpax.tex]{doc-extract-newpax.tex}
77
78 Running this document will create the files \file{doc-input1.newpax} and \file{doc-input2.newpax}.
79
80 To find the graphics \texttt{kpathsea} is used. This means that graphics in texmf trees will work and
81 you can also use paths to directories, but settings in \cs{graphicspath} are ignored. The newpax
82 file is currently written into the current directory, which means that graphics with the same name
83 in different locations won't work easily (with lualatex you could create the newpax file in the document
84 just before it is needed). Later versions of the package will probably add some options for this case,
85 but for now use at best distinct file names.
86
87
88 \subsection{Step 2: Using the \texttt{.newpax}-file with \texorpdfstring{\pkg{newpax}}{newpax}}
89
90 The package \pkg{newpax} is based on the package \pkg{pax} but extends it in various way.
91 It requires the \LaTeX{} PDF management
92 code in the \pkg{pdfmanagement-testphase} package. The PDF management is \emph{not} compatible with every package, check its documentation!
93
94 The following listing shows how to use \pkg{newpax}.
95
96 \begin{itemize}
97 \item It should work with pdflatex, lualatex and xelatex. The latex/dvips route fails as this can't include PDF anyway.
98 \item Some provision have been added to allow multiple inclusion of the same PDF, but if you insert different sets of pages from a PDF some destinations can still be missing. So better avoid it.
99 \item You can choose for every file if border color and styles of links are taken from the source PDF or from the hyperref settings. But you can't adjust or change colored links.
100 \item You can add additional settings to the annotations, for example an \texttt{/F} flag, with
101
102 \begin{lstlisting}
103 \ExplSyntaxOn
104 \pdfannot_dict_put:nnn {link/URI}{F}{4}
105 \ExplSyntaxOff
106 \end{lstlisting}
107
108 \end{itemize}
109
110 \lstinputlisting[firstline=2,caption=doc-use-newpax.tex]{doc-use-newpax.tex}
111
112 \subsection{Combining the steps}
113
114 When using lualatex both step can be simply in the same document.
115 With other engines you can use \pkg{ifluatex}.
116
117
118
119 \section{Setup options}
120
121 \DescribeMacro\newpaxsetup{key-val option list}
122
123 This command allows to change the behaviour inclusion. It knows the following keys:
124
125 \begin{description}
126 \item[\PrintKeyName{usefileattributes}] This is a boolean key. If set to true, the reinserted annotations will use the linkborder settings (color and style) of the included file, if set to false, the settings of the receiving PDF will take precendence.
127
128 \item[\PrintKeyName{destsuffix}] This allows to add a suffix to the destination names. This is needed if a file with destinations
129 is included more than once, to avoid to get multiple destinations.
130
131 \item[\PrintKeyName{addannots}] This is a boolean key. It allows to switch on and off the reinserting of the annotations. When set to false it also suppress warnings in the log if the \file{.newpax} file is not found.
132 It is recommended to set it to false for graphics which don't have links.
133
134 \item[\PrintKeyName{dests}] This is a choice key. Currently the values
135 \texttt{used} (the default) and \texttt{all} are allowed. In the first
136 case only destinations that are targets of links in the included PDF
137 will be included, in the second case all destinations (if they are in
138 the included pages) will be included. The second can be useful if you
139 want to link to destinations \enquote{from the outside}, see below
140 section~\ref{sec:outside}.
141 \end{description}
142
143
144 \section{More Background}
145
146 Clickable links in a PDF are one example of an annotation. Annotations are areas on a page which are associated with an action. A typical annotation object could look like this in the PDF:
147
148 \begin{lstlisting}
149 15 0 obj
150 <<
151 /Type /Annot
152 /Subtype/Link
153 /Rect [147.716 654.025 301.887 665.15]
154 /Border[0 0 1]/BS<</S/U/W 1>>/H/I/C[0 1 1]
155 /A<</Type/Action/S/URI/URI(https://www.latex-project.org)>>
156 >>
157 endobj
158 \end{lstlisting}
159 This is an object of type \texttt{Annot} and subtype \texttt{Link}.
160 The \texttt{/Rect} value describes the rectangle of this annotation. The coordinates are absolute coordinates related to the current page. It is important to understand that an annotation is not connected to some page content but only to a location!
161 The \texttt{/Border} setting and the other values in this line describe the look and color of annotation. The \texttt{/A} value contains the action, in this case it is an url to an external website.
162
163
164 To \enquote{reactivate} the annotations of an included pdf one has to do a number of tasks.
165 \begin{itemize}
166 \item One must \emph{retrieve and store} the annotations of the included pdf. For links to external url's this requires to find only one object like the one shown above. But e.g. internal links point to destination objects and these must be found too.
167 \item One must \emph{recalculate} the rectangle coordinates to fit to the coordinate system of the target page: as the included pdf can be placed at various positions, scaled, rotated and even clipped this is not an easy task. Destinations have rectangles too that must be recalculated.
168 \item One must \emph{reinsert} the annotation and related objects. This has to take into account that a pdf is perhaps not included completely, a link shouldn't point to a missing page or a clipped annotation. It also has to take into account that a pdf is perhaps inserted more than once or in steps.
169 \end{itemize}
170
171 \subsection{Retrieving and storing annotations}
172
173 Theoretically one can do it manually: Uncompress the PDF (or when using \LaTeX, create directly an uncompressed one), open it in an editor and copy and paste all needed objects. Practically one naturally want some tool.
174
175 The \pkg{pax} package from Heiko Oberdiek consists of a perl script and a java-jar file \texttt{PDFAnnotExtractor} which can extract the necessary objects. It writes the information to a file with the extension \texttt{pax}.
176 When it has been successfully installed it works quite fine. Problems with this approach are
177 \begin{itemize}
178 \item \texttt{PDFAnnotExtractor} requires an external, old version of the java library of PDFbox which must be installed manually;
179 \item it requires a java installation and
180 \item it is not extensible.
181 \end{itemize}
182
183 The \pkg{newpax} package comes with a lua-file. It uses the \texttt{pdfe} library embedded in luatex to extract the annotations and other needed information. \pkg{newpax} writes the information to a file with the extension \texttt{pax} or \texttt{newpax}. The content of the files is (nearly) identical to the content of the \file{pax}-file written by \texttt{PDFAnnotExtractor}. The lua code was written by looking at example outputs from \texttt{PDFAnnotExtractor} and reproducing it in lua. The ordering of some elements is a bit different and some strings are output in a different way but for the examples I used the resulting \texttt{pax}-files can be used together with the original \texttt{pax.sty}. But due to the fact that the code was written without real spec simply by looking at examples, it is quite probably that the lua code is not yet handling all objects or options that \texttt{PDFAnnotExtractor} outputs. But the code can rather easily be extended when the needs arises.
184
185 The code also doesn't handle structure elements, neither at the export nor at the import. I have yet no real idea what would be sensible here (and I'm quite sure that \texttt{PDFAnnotExtractor} doesn't handle this either.)
186
187
188 \section{Importing annotations}
189
190 The import function has to handle two main problems:
191 \begin{itemize}
192 \item Recalculation of coordinates if the imported PDF is scaled or moved
193 \item Dropping of annotations and destinations if a PDF is only partially included, e.g. because the graphic is clipped, or because only a selection of pages are included.
194 \end{itemize}
195
196 The \pkg{pax} package from Heiko Oberdiek does here hard work to recalculate the annotation rectangles and to decide which annotation and which destination should be reinserted and patches \cs{includegraphics} command to automate this. \pkg{newpax} mostly reuses the core commands of \pkg{pax} and added only a number of switches and support for more engines and backends.
197
198 \section{Internal links}
199
200 Internal links (GoTo links in PDF speach) are more complicated than the other link types.
201
202 At first they involve two objects: the link annotation and the target of the link (destination in PDF speach). If a PDF is included partially, it is therefore not enough to check if the link area is on the visible pages, one also has to check if the target of the link is there. As such a target can be on a later page the \pkg{pax}/\pkg{newpax} uses the \texttt{.aux} file to record which targets exists and which are required and in a second compilation decides which links and destinations should be reinserted. As an example: if you include from a PDF the table of contents (with links) and a few pages, only the links in the toc pointing to visible sections \emph{and} only the destinations needed for this links will be reinserted.
203
204 At second the targets are normally „named destinations“, that means a link
205 annotation points to a string like \texttt{section.1} and the name tree
206 \texttt{/Dests} contains a mapping for this string to an destination object.
207 As names like \texttt{section.1} are used in many PDFs produced by \LaTeX{}
208 they can not simply be reused when reinserting annotations. \pkg{newpax}
209 tries to avoid name clashes by generating names consisting of a prefix with
210 the file name and number or name. So e.g. the \texttt{section.1} destination
211 would be called \texttt{[email protected]} in the receiving file. If you
212 set up a suffix, e.g. with \texttt{destsuffix=A}, it is appended with an @
213 symbol, so the result would be \texttt{[email protected]@A}. If the
214 original destination has not name (this can happen if you include PDFs which
215 haven't been created by TeX), then a number is used, so you get
216 \texttt{file.newpax@1}.
217
218
219 \subsection{Access from the „outside“}\label{sec:outside}
220
221 It is possible to link from the external document to destinations in the
222 included PDF, for example to build a table of contents with links. For this
223 you need
224
225 \begin{itemize}
226 \item the included PDF must contain all destinations that you want to use.
227 That means, \pkg{newpax} won't create destinations out of the blue.
228
229 \item You must ensure that the destinations you need are imported. This
230 will be the case if they are targets of internal links in the imported
231 PDF, if not you can force that all destinations of visible pages are
232 there by using the option \texttt{dests=all}, see above.
233
234 \item You need the names of the destinations. If the imported PDF has been
235 created with \LaTeX{} you can look in the \texttt{toc} or
236 \texttt{aux}-file to find names. Then you can setup a filter to link to
237 the names used by \pkg{newpax} and e.g. load the toc or copy some of
238 the content lines:
239
240 \begin{lstlisting}
241 \newpaxsetup{dests=all}
242 \def\HyperDestNameFilter#1{myinput.newpax@#1}
243 \input{myinput.toc} %load toc
244 \includepdf[pages={2-4}]{myinput}
245 \end{lstlisting}
246
247 \end{itemize}
248
249
250
251
252
253
254 \section{Example input}
255
256 \fbox{\includegraphics[scale=0.5,trim=4cm 15cm 8cm 3cm,clip,page=1]{doc-input1}}
257 \fbox{\includegraphics[scale=0.5,trim=5cm 15cm 8cm 3cm,clip,page=2]{doc-input1}}
258
259 Check also the output of the listing above, \file{doc-use-newpax.pdf}.
260
261 \section{Support for the \texorpdfstring{\pkg{pax}}{pax} package}
262
263 \subsection{Step 1: Extracting the annotations}
264 The lua script is also able to write \texttt{pax} files for the \pkg{pax} package (and so can be used to replace the java application).
265
266 For this extract the annotations like this:
267
268 \lstinputlisting[caption=doc-extract-pax.tex]{doc-extract-pax.tex}
269
270 \subsection{Step 2: Using the \texttt{.pax}-file with \pkg{pax.sty}}
271
272 Ensure that the \texttt{.pax} file created in step 1 can be found by your main document. You can then insert your PDF files together with their annotations like in the following listing.
273
274 \begin{itemize}
275 \item This works with pdflatex and lualatex. lualatex needs the extra code demonstrated in the document.
276 \item It needs two or three compilations until every reference is correct.
277 \item There is a small typo in \pkg{pax.sty} which affects clipping, the patch shown in the listing correct this.
278 \item In some cases the catcode of \# and \% must be set to letter to avoid errors.
279 \item Don't include PDFs with destinations twice as this will lead to duplicate destinations and pdflatex will complain.
280 \item If annotations should not be reinserted remove the \texttt{.pax}-file.
281 \item If \pkg{hyperref} is loaded you can change the color and style of link borders with hyperref options.
282 \end{itemize}
283 \enlargethispage{\baselineskip}
284 \lstinputlisting[firstline=2,caption=doc-use-pax.tex,escapechar={}]{doc-use-pax.tex}
285
286 \end{document}