Debian Codegraph @

texmf-dist/doc/latex/l3kernel/l3term-glossary.tex

raw
1 \iffalse meta-comment
2
3 File: l3term-glossary.tex
4
5 Copyright (C) 2018-2026 The LaTeX Project
6
7 It may be distributed and/or modified under the conditions of the
8 LaTeX Project Public License (LPPL), either version 1.3c of this
9 license or (at your option) any later version. The latest version
10 of this license is in the file
11
12 https://www.latex-project.org/lppl.txt
13
14 This file is part of the "l3kernel bundle" (The Work in LPPL)
15 and all files in that bundle must be distributed together.
16
17 The released version of this bundle is available from CTAN.
18
19 \fi
20
21 \documentclass{l3doc}
22
23
24 \title{%
25 Glossary of \TeX{} terms used to describe \LaTeX3 functions%
26 }
27 \author{%
28 The \LaTeX{} Project\thanks
29 {%
30 E-mail:
31 \href{mailto:[email protected]}%
32 {[email protected]}%
33 }%
34 }
35 \date{Released 2026-05-15}
36
37 \newcommand{\TF}{\textit{(TF)}}
38
39 \begin{document}
40
41 \maketitle
42
43 This file describes aspects of \TeX{} programming that are relevant in a
44 \pkg{expl3} context.
45
46 \section{Reading a file}
47
48 Tokenization.
49
50 Treatment of spaces, such as the trap that \verb|\~~a| is equivalent to
51 \verb|\~a| in \pkg{expl3} syntax, or that \verb|~| fails to give a space at the
52 beginning of a line.
53
54 \section{Structure of tokens}
55
56 We refer to the documentation of \texttt{l3token} for a complete
57 description of all \TeX{} tokens. We distinguish the meaning of the
58 token, which controls the expansion of the token and its effect on
59 \TeX{}'s state, and its shape, which is used when comparing token lists
60 such as for delimited arguments. At any given time two tokens of the
61 same shape automatically have the same meaning, but the converse does
62 not hold, and the meaning associated with a given shape change when
63 doing assignments.
64
65 Apart from a few exceptions, a token has one of the following shapes.
66 \begin{itemize}
67 \item A control sequence, characterized by the sequence of characters
68 that constitute its name: for instance, \cs{use:n} is a five-letter
69 control sequence.
70 \item An active character token, characterized by its character code
71 (between $0$ and $1114111$ for \LuaTeX{} and \XeTeX{} and less for
72 other engines) and category code~$13$.
73 \item A character token such as |A| or |#|, characterized by its
74 character code and category code (one of $1$, $2$, $3$, $4$, $6$,
75 $7$, $8$, $10$, $11$ or~$12$ whose meaning is described below).
76 \end{itemize}
77
78 The meaning of a (non-active) character token is fixed by its category
79 code (and character code) and cannot be changed. We call these tokens
80 \emph{explicit} character tokens. Category codes that a character token
81 can have are listed below by giving a sample output of the \TeX{}
82 primitive \tn{meaning}, together with their \pkg{expl3} names and most
83 common example:
84 \begin{itemize}
85 \item[1] begin-group character (|group_begin|, often |{|),
86 \item[2] end-group character (|group_end|, often |}|),
87 \item[3] math shift character (|math_toggle|, often |$|), % $
88 \item[4] alignment tab character (|alignment|, often |&|),
89 \item[6] macro parameter character (|parameter|, often |#|),
90 \item[7] superscript character (|math_superscript|, often |^|),
91 \item[8] subscript character (|math_subscript|, often |_|),
92 \item[10] blank space (|space|, often character code~$32$),
93 \item[11] the letter (|letter|, such as |A|),
94 \item[12] the character (|other|, such as |0|).
95 \end{itemize}
96 Category code~$13$ (|active|) is discussed below. Input characters can
97 also have several other category codes which do not lead to character
98 tokens for later processing: $0$~(|escape|), $5$~(|end_line|),
99 $9$~(|ignore|), $14$~(|comment|), and $15$~(|invalid|).
100
101 The meaning of a control sequence or active character can be identical
102 to that of any character token listed above (with any character code),
103 and we call such tokens \emph{implicit} character tokens. The meaning
104 is otherwise in the following list:
105 \begin{itemize}
106 \item a macro, used in \pkg{expl3} for most functions and some variables
107 (|tl|, |fp|, |seq|, \ldots{}),
108 \item a primitive such as \tn{def} or \tn{topmark}, used in \pkg{expl3}
109 for some functions,
110 \item a register such as \tn{count}|123|, used in \pkg{expl3} for the
111 implementation of some variables (|int|, |dim|, \ldots{}),
112 \item a constant integer such as \tn{char}|"56| or
113 \tn{mathchar}|"121|, used when defining a constant using
114 \cs{int_const:Nn},
115 \item a font selection command,
116 \item undefined.
117 \end{itemize}
118 Macros can be \tn{protected} or not, \tn{long} or not (the opposite of
119 what \pkg{expl3} calls |nopar|), and \tn{outer} or not (unused in \pkg{expl3}).
120 Their \tn{meaning} takes the form
121 \begin{quote}
122 \meta{prefix} |macro:|\meta{argument}|->|\meta{replacement}
123 \end{quote}
124 where \meta{prefix} is among \tn{protected}\tn{long}\tn{outer},
125 \meta{argument} describes parameters that the macro expects, such as
126 |#1#2#3|, and \meta{replacement} describes how the parameters are
127 manipulated, such as~|\int_eval:n{#2+#1*#3}|. This information can be
128 accessed by \cs{cs_prefix_spec:N}, \cs{cs_parameter_spec:N},
129 \cs{cs_replacement_spec:N}.
130
131 When a macro takes an undelimited argument, explicit space characters
132 (with character code $32$ and category code $10$) are ignored. If the
133 following token is an explicit character token with category code $1$
134 (begin-group) and an arbitrary character code, then \TeX{} scans ahead
135 to obtain an equal number of explicit character tokens with category
136 code $1$ (begin-group) and $2$ (end-group), and the resulting list of
137 tokens (with outer braces removed) becomes the argument. Otherwise, a
138 single token is taken as the argument for the macro: we call such single
139 tokens \enquote{N-type}, as they are suitable to be used as an argument
140 for a function with the signature~\texttt{:N}.
141
142 When a macro takes a delimited argument \TeX{} scans ahead until finding
143 the delimiter (outside any pairs of begin-group/end-group explicit
144 characters), and the resulting list of tokens (with outer braces
145 removed) becomes the argument. Note that explicit space characters at
146 the start of the argument are \emph{not} ignored in this case (and they
147 prevent brace-stripping).
148
149 \section{Handling of hash tokens}
150
151 \TeX{} uses the hash (octothorpe) character |#| to denote parameters for
152 macros: these must be numbered sequentially. To allow handling of nested
153 macros, \TeX{} requires that for each nesting level, hash tokens are doubled.
154 For example
155 \begin{verbatim}
156 \cs_new:Npn \mypkg_outer:N #1
157 {
158 \cs_new:Npn \mypkg_inner:N ##1
159 {
160 #1
161 ##1
162 }
163 }
164 \end{verbatim}
165 would define both |\mypkg_outer:N| and |\mypkg_inner:N| as taking
166 exactly one argument. If we then do
167 \begin{verbatim}
168 \mypkg_outer:N \foo
169 \cs_show:N \mypkg_inner:N
170 \end{verbatim}
171 \TeX{} will report
172 \begin{verbatim}
173 > \mypkg_inner:N=\long macro:#1->\foo #1.
174 \end{verbatim}
175 i.e., the hash is not doubled, but is now the parameter of this macro.
176
177 Exactly the same concept applies to anywhere that inline code is nested in
178 \pkg{expl3}, for example inline mapping code, key definitions, etc.
179
180 \end{document}