Debian Codegraph @

llvm/docs/TableGen/index.rst

raw
1 =================
2 TableGen Overview
3 =================
4
5 .. contents::
6 :local:
7
8 .. toctree::
9 :hidden:
10
11 BackEnds
12 BackGuide
13 ProgRef
14
15 Introduction
16 ============
17
18 TableGen's purpose is to help a human develop and maintain records of
19 domain-specific information. Because there may be a large number of these
20 records, it is specifically designed to allow writing flexible descriptions and
21 for common features of these records to be factored out. This reduces the
22 amount of duplication in the description, reduces the chance of error, and makes
23 it easier to structure domain-specific information.
24
25 The TableGen front end parses a file, instantiates the declarations, and
26 hands the result off to a domain-specific `backend`_ for processing. See
27 the :doc:`TableGen Programmer's Reference <./ProgRef>` for an in-depth
28 description of TableGen. See :doc:`tblgen - Description to C++ Code
29 <../CommandGuide/tblgen>` for details on the ``*-tblgen`` commands
30 that run the various flavors of TableGen.
31
32 The current major users of TableGen are :doc:`The LLVM Target-Independent
33 Code Generator <../CodeGenerator>` and the `Clang diagnostics and attributes
34 <https://clang.llvm.org/docs/UsersManual.html#controlling-errors-and-warnings>`_.
35
36 Note that if you work with TableGen frequently and use emacs or vim,
37 you can find an emacs "TableGen mode" and a vim language file in the
38 ``llvm/utils/emacs`` and ``llvm/utils/vim`` directories of your LLVM
39 distribution, respectively.
40
41 .. _intro:
42
43
44 The TableGen program
45 ====================
46
47 TableGen files are interpreted by the TableGen program: ``llvm-tblgen`` available
48 in your build directory under ``bin``. It is not installed in the system (or where
49 your sysroot is set to), since it has no use beyond LLVM's build process.
50
51 Running TableGen
52 ----------------
53
54 TableGen runs just like any other LLVM tool. The first (optional) argument
55 specifies the file to read. If a filename is not specified, ``llvm-tblgen``
56 reads from standard input.
57
58 The ``-o`` option specifies the output file or ``-`` to output to
59 stdout. Where TableGen produces multiple output files, the option
60 specifies the name of the main output file, which also works as the
61 name prefix for other output files.
62
63 To be useful, one of the `backends`_ must be used. These backends are
64 selectable on the command line (type '``llvm-tblgen -help``' for a list). For
65 example, to get a list of all of the definitions that subclass a particular type
66 (which can be useful for building up an enum list of these records), use the
67 ``-print-enums`` option:
68
69 .. code-block:: bash
70
71 $ llvm-tblgen X86.td -print-enums -class=Register
72 AH, AL, AX, BH, BL, BP, BPL, BX, CH, CL, CX, DH, DI, DIL, DL, DX, EAX, EBP, EBX,
73 ECX, EDI, EDX, EFLAGS, EIP, ESI, ESP, FP0, FP1, FP2, FP3, FP4, FP5, FP6, IP,
74 MM0, MM1, MM2, MM3, MM4, MM5, MM6, MM7, R10, R10B, R10D, R10W, R11, R11B, R11D,
75 R11W, R12, R12B, R12D, R12W, R13, R13B, R13D, R13W, R14, R14B, R14D, R14W, R15,
76 R15B, R15D, R15W, R8, R8B, R8D, R8W, R9, R9B, R9D, R9W, RAX, RBP, RBX, RCX, RDI,
77 RDX, RIP, RSI, RSP, SI, SIL, SP, SPL, ST0, ST1, ST2, ST3, ST4, ST5, ST6, ST7,
78 XMM0, XMM1, XMM10, XMM11, XMM12, XMM13, XMM14, XMM15, XMM2, XMM3, XMM4, XMM5,
79 XMM6, XMM7, XMM8, XMM9,
80
81 $ llvm-tblgen X86.td -print-enums -class=Instruction
82 ABS_F, ABS_Fp32, ABS_Fp64, ABS_Fp80, ADC32mi, ADC32mi8, ADC32mr, ADC32ri,
83 ADC32ri8, ADC32rm, ADC32rr, ADC64mi32, ADC64mi8, ADC64mr, ADC64ri32, ADC64ri8,
84 ADC64rm, ADC64rr, ADD16mi, ADD16mi8, ADD16mr, ADD16ri, ADD16ri8, ADD16rm,
85 ADD16rr, ADD32mi, ADD32mi8, ADD32mr, ADD32ri, ADD32ri8, ADD32rm, ADD32rr,
86 ADD64mi32, ADD64mi8, ADD64mr, ADD64ri32, ...
87
88 The default backend prints out all of the records. There is also a general
89 backend which outputs all the records as a JSON data structure, enabled using
90 the `-dump-json` option.
91
92 If you plan to use TableGen, you will most likely have to write a `backend`_
93 that extracts the information specific to what you need and formats it in the
94 appropriate way. You can do this by extending TableGen itself in C++ or by
95 writing a script in any language that can consume the JSON output.
96
97 Example
98 -------
99
100 With no other arguments, `llvm-tblgen` parses the specified file and prints out all
101 of the classes, then all of the definitions. This is a good way to see what the
102 various definitions expand to fully. Running this on the ``X86.td`` file prints
103 this (at the time of this writing):
104
105 .. code-block:: text
106
107 ...
108 def ADD32rr { // Instruction X86Inst I
109 string Namespace = "X86";
110 dag OutOperandList = (outs GR32:$dst);
111 dag InOperandList = (ins GR32:$src1, GR32:$src2);
112 string AsmString = "add{l}\t{$src2, $dst|$dst, $src2}";
113 list<dag> Pattern = [(set GR32:$dst, (add GR32:$src1, GR32:$src2))];
114 list<Register> Uses = [];
115 list<Register> Defs = [EFLAGS];
116 list<Predicate> Predicates = [];
117 int CodeSize = 3;
118 int AddedComplexity = 0;
119 bit isReturn = 0;
120 bit isBranch = 0;
121 bit isIndirectBranch = 0;
122 bit isBarrier = 0;
123 bit isCall = 0;
124 bit canFoldAsLoad = 0;
125 bit mayLoad = 0;
126 bit mayStore = 0;
127 bit isImplicitDef = 0;
128 bit isConvertibleToThreeAddress = 1;
129 bit isCommutable = 1;
130 bit isTerminator = 0;
131 bit isReMaterializable = 0;
132 bit isPredicable = 0;
133 bit hasDelaySlot = 0;
134 bit usesCustomInserter = 0;
135 bit hasCtrlDep = 0;
136 bit isNotDuplicable = 0;
137 bit hasSideEffects = 0;
138 InstrItinClass Itinerary = NoItinerary;
139 string Constraints = "";
140 string DisableEncoding = "";
141 bits<8> Opcode = { 0, 0, 0, 0, 0, 0, 0, 1 };
142 Format Form = MRMDestReg;
143 bits<6> FormBits = { 0, 0, 0, 0, 1, 1 };
144 ImmType ImmT = NoImm;
145 bits<3> ImmTypeBits = { 0, 0, 0 };
146 bit hasOpSizePrefix = 0;
147 bit hasAdSizePrefix = 0;
148 bits<4> Prefix = { 0, 0, 0, 0 };
149 bit hasREX_WPrefix = 0;
150 FPFormat FPForm = ?;
151 bits<3> FPFormBits = { 0, 0, 0 };
152 }
153 ...
154
155 This definition corresponds to the 32-bit register-register ``add`` instruction
156 of the x86 architecture. ``def ADD32rr`` defines a record named
157 ``ADD32rr``, and the comment at the end of the line indicates the superclasses
158 of the definition. The body of the record contains all of the data that
159 TableGen assembled for the record, indicating that the instruction is part of
160 the ``X86`` namespace, the pattern indicating how the instruction is selected by
161 the code generator, that it is a two-address instruction, has a particular
162 encoding, etc. The contents and semantics of the information in the record are
163 specific to the needs of the X86 backend, and are only shown as an example.
164
165 As you can see, a lot of information is needed for every instruction supported
166 by the code generator, and specifying it all manually would be unmaintainable,
167 prone to bugs, and tiring to do in the first place. Because we are using
168 TableGen, all of the information was derived from the following definition:
169
170 .. code-block:: text
171
172 let Defs = [EFLAGS],
173 isCommutable = 1, // X = ADD Y,Z --> X = ADD Z,Y
174 isConvertibleToThreeAddress = 1 in // Can transform into LEA.
175 def ADD32rr : I<0x01, MRMDestReg, (outs GR32:$dst),
176 (ins GR32:$src1, GR32:$src2),
177 "add{l}\t{$src2, $dst|$dst, $src2}",
178 [(set GR32:$dst, (add GR32:$src1, GR32:$src2))]>;
179
180 This definition makes use of the custom class ``I`` (extended from the custom
181 class ``X86Inst``), which is defined in the X86-specific TableGen file, to
182 factor out the common features that instructions of its class share. A key
183 feature of TableGen is that it allows the end user to define the abstractions
184 they prefer to use when describing their information.
185
186 Syntax
187 ======
188
189 TableGen has a syntax loosely based on C++ templates, with built-in
190 types and specification. In addition, TableGen's syntax introduces some
191 automation concepts like multiclass, foreach, let, etc.
192
193 Basic concepts
194 --------------
195
196 TableGen files consist of two key parts: 'classes' and 'definitions', both of
197 which are considered 'records'.
198
199 **TableGen records** have a unique name, a list of values, and a list of
200 superclasses. The list of values is the main data that TableGen builds for each
201 record; it is this that holds the domain-specific information for the
202 application. The interpretation of this data is left to a specific `backend`_,
203 but the structure and format rules are taken care of and fixed by
204 TableGen.
205
206 **TableGen definitions** are the concrete form of 'records'. These generally do
207 not have any undefined values and are marked with the '``def``' keyword.
208
209 .. code-block:: text
210
211 def FeatureFPARMv8 : SubtargetFeature<"fp-armv8", "HasFPARMv8", "true",
212 "Enable ARMv8 FP">;
213
214 In this example, ``FeatureFPARMv8`` is ``SubtargetFeature`` record initialised
215 with some values. The names of the classes are defined via the
216 keyword `class` either in the same file or some other included. Most target
217 TableGen files include the generic ones in ``include/llvm/Target``.
218
219 **TableGen classes** are abstract records that are used to build and describe
220 other records. These classes allow the end-user to build abstractions for
221 either the domain they are targeting (such as ``Register``, ``RegisterClass``, and
222 ``Instruction`` in the LLVM code generator) or for the implementor to help factor
223 out common properties of records (such as ``FPInst``, which is used to represent
224 floating point instructions in the X86 backend). TableGen keeps track of all of
225 the classes that are used to build up a definition, so the backend can find all
226 definitions of a particular class, such as ``Instruction``.
227
228 .. code-block:: text
229
230 class ProcNoItin<string Name, list<SubtargetFeature> Features>
231 : Processor<Name, NoItineraries, Features>;
232
233 Here, the class ``ProcNoItin``, receiving parameters ``Name`` of type ``string`` and
234 a list of target features is specializing the class ``Processor`` by passing the
235 arguments down as well as hard-coding ``NoItineraries``.
236
237 **TableGen multiclasses** are groups of abstract records that are instantiated
238 all at once. Each instantiation can result in multiple TableGen definitions.
239 If a multiclass inherits from another multiclass, the definitions in the
240 sub-multiclass become part of the current multiclass, as if they were declared
241 in the current multiclass.
242
243 .. code-block:: text
244
245 multiclass ro_signed_pats<string T, string Rm, dag Base, dag Offset, dag Extend,
246 dag address, ValueType sty> {
247 def : Pat<(i32 (!cast<SDNode>("sextload" # sty) address)),
248 (!cast<Instruction>("LDRS" # T # "w_" # Rm # "_RegOffset")
249 Base, Offset, Extend)>;
250
251 def : Pat<(i64 (!cast<SDNode>("sextload" # sty) address)),
252 (!cast<Instruction>("LDRS" # T # "x_" # Rm # "_RegOffset")
253 Base, Offset, Extend)>;
254 }
255
256 defm : ro_signed_pats<"B", Rm, Base, Offset, Extend,
257 !foreach(decls.pattern, address,
258 !subst(SHIFT, imm_eq0, decls.pattern)),
259 i8>;
260
261 See the :doc:`TableGen Programmer's Reference <./ProgRef>` for an in-depth
262 description of TableGen.
263
264
265 .. _backend:
266 .. _backends:
267
268 TableGen backends
269 =================
270
271 TableGen files have no real meaning without a backend. The default operation
272 when running ``*-tblgen`` is to print the information in a textual format, but
273 that's only useful for debugging the TableGen files themselves. The power
274 in TableGen is, however, to interpret the source files into an internal
275 representation that can be generated into anything you want.
276
277 Current usage of TableGen is to create huge include files with tables that you
278 can either include directly (if the output is in the language you're coding),
279 or be used in pre-processing via macros surrounding the include of the file.
280
281 Direct output can be used if the backend already prints a table in C format
282 or if the output is just a list of strings (for error and warning messages).
283 Pre-processed output should be used if the same information needs to be used
284 in different contexts (like Instruction names), so your backend should print
285 a meta-information list that can be shaped into different compile-time formats.
286
287 See :doc:`TableGen BackEnds <./BackEnds>` for a list of available
288 backends, and see the :doc:`TableGen Backend Developer's Guide <./BackGuide>`
289 for information on how to write and debug a new backend.
290
291 Tools and Resources
292 ===================
293
294 In addition to this documentation, a list of tools and resources for TableGen
295 can be found in TableGen's
296 `README <https://github.com/llvm/llvm-project/blob/main/llvm/utils/TableGen/README.md>`_.
297
298 TableGen Deficiencies
299 =====================
300
301 Despite being very generic, TableGen has some deficiencies that have been
302 pointed out numerous times. The common theme is that, while TableGen allows
303 you to build domain-specific languages, the final languages that you create
304 lack the power of other DSLs, which in turn considerably increases the size
305 and complexity of TableGen files.
306
307 At the same time, TableGen allows you to create virtually any meaning of
308 the basic concepts via custom-made backends, which can pervert the original
309 design and make it very hard for newcomers to understand the evil TableGen
310 file.
311
312 There are some in favor of extending the semantics even more, but making sure
313 backends adhere to strict rules. Others suggest moving to fewer,
314 more powerful DSLs designed with specific purposes, or even reusing existing
315 DSLs.