| 1 |
================= |
| 2 |
TableGen Overview |
| 3 |
================= |
| 4 |
|
| 5 |
.. contents:: |
| 6 |
:local: |
| 7 |
|
| 8 |
.. toctree:: |
| 9 |
:hidden: |
| 10 |
|
| 11 |
BackEnds |
| 12 |
BackGuide |
| 13 |
ProgRef |
| 14 |
|
| 15 |
Introduction |
| 16 |
============ |
| 17 |
|
| 18 |
TableGen's purpose is to help a human develop and maintain records of |
| 19 |
domain-specific information. Because there may be a large number of these |
| 20 |
records, it is specifically designed to allow writing flexible descriptions and |
| 21 |
for common features of these records to be factored out. This reduces the |
| 22 |
amount of duplication in the description, reduces the chance of error, and makes |
| 23 |
it easier to structure domain-specific information. |
| 24 |
|
| 25 |
The TableGen front end parses a file, instantiates the declarations, and |
| 26 |
hands the result off to a domain-specific `backend`_ for processing. See |
| 27 |
the :doc:`TableGen Programmer's Reference <./ProgRef>` for an in-depth |
| 28 |
description of TableGen. See :doc:`tblgen - Description to C++ Code |
| 29 |
<../CommandGuide/tblgen>` for details on the ``*-tblgen`` commands |
| 30 |
that run the various flavors of TableGen. |
| 31 |
|
| 32 |
The current major users of TableGen are :doc:`The LLVM Target-Independent |
| 33 |
Code Generator <../CodeGenerator>` and the `Clang diagnostics and attributes |
| 34 |
<https://clang.llvm.org/docs/UsersManual.html#controlling-errors-and-warnings>`_. |
| 35 |
|
| 36 |
Note that if you work with TableGen frequently and use emacs or vim, |
| 37 |
you can find an emacs "TableGen mode" and a vim language file in the |
| 38 |
``llvm/utils/emacs`` and ``llvm/utils/vim`` directories of your LLVM |
| 39 |
distribution, respectively. |
| 40 |
|
| 41 |
.. _intro: |
| 42 |
|
| 43 |
|
| 44 |
The TableGen program |
| 45 |
==================== |
| 46 |
|
| 47 |
TableGen files are interpreted by the TableGen program: ``llvm-tblgen`` available |
| 48 |
in your build directory under ``bin``. It is not installed in the system (or where |
| 49 |
your sysroot is set to), since it has no use beyond LLVM's build process. |
| 50 |
|
| 51 |
Running TableGen |
| 52 |
---------------- |
| 53 |
|
| 54 |
TableGen runs just like any other LLVM tool. The first (optional) argument |
| 55 |
specifies the file to read. If a filename is not specified, ``llvm-tblgen`` |
| 56 |
reads from standard input. |
| 57 |
|
| 58 |
The ``-o`` option specifies the output file or ``-`` to output to |
| 59 |
stdout. Where TableGen produces multiple output files, the option |
| 60 |
specifies the name of the main output file, which also works as the |
| 61 |
name prefix for other output files. |
| 62 |
|
| 63 |
To be useful, one of the `backends`_ must be used. These backends are |
| 64 |
selectable on the command line (type '``llvm-tblgen -help``' for a list). For |
| 65 |
example, to get a list of all of the definitions that subclass a particular type |
| 66 |
(which can be useful for building up an enum list of these records), use the |
| 67 |
``-print-enums`` option: |
| 68 |
|
| 69 |
.. code-block:: bash |
| 70 |
|
| 71 |
$ llvm-tblgen X86.td -print-enums -class=Register |
| 72 |
AH, AL, AX, BH, BL, BP, BPL, BX, CH, CL, CX, DH, DI, DIL, DL, DX, EAX, EBP, EBX, |
| 73 |
ECX, EDI, EDX, EFLAGS, EIP, ESI, ESP, FP0, FP1, FP2, FP3, FP4, FP5, FP6, IP, |
| 74 |
MM0, MM1, MM2, MM3, MM4, MM5, MM6, MM7, R10, R10B, R10D, R10W, R11, R11B, R11D, |
| 75 |
R11W, R12, R12B, R12D, R12W, R13, R13B, R13D, R13W, R14, R14B, R14D, R14W, R15, |
| 76 |
R15B, R15D, R15W, R8, R8B, R8D, R8W, R9, R9B, R9D, R9W, RAX, RBP, RBX, RCX, RDI, |
| 77 |
RDX, RIP, RSI, RSP, SI, SIL, SP, SPL, ST0, ST1, ST2, ST3, ST4, ST5, ST6, ST7, |
| 78 |
XMM0, XMM1, XMM10, XMM11, XMM12, XMM13, XMM14, XMM15, XMM2, XMM3, XMM4, XMM5, |
| 79 |
XMM6, XMM7, XMM8, XMM9, |
| 80 |
|
| 81 |
$ llvm-tblgen X86.td -print-enums -class=Instruction |
| 82 |
ABS_F, ABS_Fp32, ABS_Fp64, ABS_Fp80, ADC32mi, ADC32mi8, ADC32mr, ADC32ri, |
| 83 |
ADC32ri8, ADC32rm, ADC32rr, ADC64mi32, ADC64mi8, ADC64mr, ADC64ri32, ADC64ri8, |
| 84 |
ADC64rm, ADC64rr, ADD16mi, ADD16mi8, ADD16mr, ADD16ri, ADD16ri8, ADD16rm, |
| 85 |
ADD16rr, ADD32mi, ADD32mi8, ADD32mr, ADD32ri, ADD32ri8, ADD32rm, ADD32rr, |
| 86 |
ADD64mi32, ADD64mi8, ADD64mr, ADD64ri32, ... |
| 87 |
|
| 88 |
The default backend prints out all of the records. There is also a general |
| 89 |
backend which outputs all the records as a JSON data structure, enabled using |
| 90 |
the `-dump-json` option. |
| 91 |
|
| 92 |
If you plan to use TableGen, you will most likely have to write a `backend`_ |
| 93 |
that extracts the information specific to what you need and formats it in the |
| 94 |
appropriate way. You can do this by extending TableGen itself in C++ or by |
| 95 |
writing a script in any language that can consume the JSON output. |
| 96 |
|
| 97 |
Example |
| 98 |
------- |
| 99 |
|
| 100 |
With no other arguments, `llvm-tblgen` parses the specified file and prints out all |
| 101 |
of the classes, then all of the definitions. This is a good way to see what the |
| 102 |
various definitions expand to fully. Running this on the ``X86.td`` file prints |
| 103 |
this (at the time of this writing): |
| 104 |
|
| 105 |
.. code-block:: text |
| 106 |
|
| 107 |
... |
| 108 |
def ADD32rr { // Instruction X86Inst I |
| 109 |
string Namespace = "X86"; |
| 110 |
dag OutOperandList = (outs GR32:$dst); |
| 111 |
dag InOperandList = (ins GR32:$src1, GR32:$src2); |
| 112 |
string AsmString = "add{l}\t{$src2, $dst|$dst, $src2}"; |
| 113 |
list<dag> Pattern = [(set GR32:$dst, (add GR32:$src1, GR32:$src2))]; |
| 114 |
list<Register> Uses = []; |
| 115 |
list<Register> Defs = [EFLAGS]; |
| 116 |
list<Predicate> Predicates = []; |
| 117 |
int CodeSize = 3; |
| 118 |
int AddedComplexity = 0; |
| 119 |
bit isReturn = 0; |
| 120 |
bit isBranch = 0; |
| 121 |
bit isIndirectBranch = 0; |
| 122 |
bit isBarrier = 0; |
| 123 |
bit isCall = 0; |
| 124 |
bit canFoldAsLoad = 0; |
| 125 |
bit mayLoad = 0; |
| 126 |
bit mayStore = 0; |
| 127 |
bit isImplicitDef = 0; |
| 128 |
bit isConvertibleToThreeAddress = 1; |
| 129 |
bit isCommutable = 1; |
| 130 |
bit isTerminator = 0; |
| 131 |
bit isReMaterializable = 0; |
| 132 |
bit isPredicable = 0; |
| 133 |
bit hasDelaySlot = 0; |
| 134 |
bit usesCustomInserter = 0; |
| 135 |
bit hasCtrlDep = 0; |
| 136 |
bit isNotDuplicable = 0; |
| 137 |
bit hasSideEffects = 0; |
| 138 |
InstrItinClass Itinerary = NoItinerary; |
| 139 |
string Constraints = ""; |
| 140 |
string DisableEncoding = ""; |
| 141 |
bits<8> Opcode = { 0, 0, 0, 0, 0, 0, 0, 1 }; |
| 142 |
Format Form = MRMDestReg; |
| 143 |
bits<6> FormBits = { 0, 0, 0, 0, 1, 1 }; |
| 144 |
ImmType ImmT = NoImm; |
| 145 |
bits<3> ImmTypeBits = { 0, 0, 0 }; |
| 146 |
bit hasOpSizePrefix = 0; |
| 147 |
bit hasAdSizePrefix = 0; |
| 148 |
bits<4> Prefix = { 0, 0, 0, 0 }; |
| 149 |
bit hasREX_WPrefix = 0; |
| 150 |
FPFormat FPForm = ?; |
| 151 |
bits<3> FPFormBits = { 0, 0, 0 }; |
| 152 |
} |
| 153 |
... |
| 154 |
|
| 155 |
This definition corresponds to the 32-bit register-register ``add`` instruction |
| 156 |
of the x86 architecture. ``def ADD32rr`` defines a record named |
| 157 |
``ADD32rr``, and the comment at the end of the line indicates the superclasses |
| 158 |
of the definition. The body of the record contains all of the data that |
| 159 |
TableGen assembled for the record, indicating that the instruction is part of |
| 160 |
the ``X86`` namespace, the pattern indicating how the instruction is selected by |
| 161 |
the code generator, that it is a two-address instruction, has a particular |
| 162 |
encoding, etc. The contents and semantics of the information in the record are |
| 163 |
specific to the needs of the X86 backend, and are only shown as an example. |
| 164 |
|
| 165 |
As you can see, a lot of information is needed for every instruction supported |
| 166 |
by the code generator, and specifying it all manually would be unmaintainable, |
| 167 |
prone to bugs, and tiring to do in the first place. Because we are using |
| 168 |
TableGen, all of the information was derived from the following definition: |
| 169 |
|
| 170 |
.. code-block:: text |
| 171 |
|
| 172 |
let Defs = [EFLAGS], |
| 173 |
isCommutable = 1, // X = ADD Y,Z --> X = ADD Z,Y |
| 174 |
isConvertibleToThreeAddress = 1 in // Can transform into LEA. |
| 175 |
def ADD32rr : I<0x01, MRMDestReg, (outs GR32:$dst), |
| 176 |
(ins GR32:$src1, GR32:$src2), |
| 177 |
"add{l}\t{$src2, $dst|$dst, $src2}", |
| 178 |
[(set GR32:$dst, (add GR32:$src1, GR32:$src2))]>; |
| 179 |
|
| 180 |
This definition makes use of the custom class ``I`` (extended from the custom |
| 181 |
class ``X86Inst``), which is defined in the X86-specific TableGen file, to |
| 182 |
factor out the common features that instructions of its class share. A key |
| 183 |
feature of TableGen is that it allows the end user to define the abstractions |
| 184 |
they prefer to use when describing their information. |
| 185 |
|
| 186 |
Syntax |
| 187 |
====== |
| 188 |
|
| 189 |
TableGen has a syntax loosely based on C++ templates, with built-in |
| 190 |
types and specification. In addition, TableGen's syntax introduces some |
| 191 |
automation concepts like multiclass, foreach, let, etc. |
| 192 |
|
| 193 |
Basic concepts |
| 194 |
-------------- |
| 195 |
|
| 196 |
TableGen files consist of two key parts: 'classes' and 'definitions', both of |
| 197 |
which are considered 'records'. |
| 198 |
|
| 199 |
**TableGen records** have a unique name, a list of values, and a list of |
| 200 |
superclasses. The list of values is the main data that TableGen builds for each |
| 201 |
record; it is this that holds the domain-specific information for the |
| 202 |
application. The interpretation of this data is left to a specific `backend`_, |
| 203 |
but the structure and format rules are taken care of and fixed by |
| 204 |
TableGen. |
| 205 |
|
| 206 |
**TableGen definitions** are the concrete form of 'records'. These generally do |
| 207 |
not have any undefined values and are marked with the '``def``' keyword. |
| 208 |
|
| 209 |
.. code-block:: text |
| 210 |
|
| 211 |
def FeatureFPARMv8 : SubtargetFeature<"fp-armv8", "HasFPARMv8", "true", |
| 212 |
"Enable ARMv8 FP">; |
| 213 |
|
| 214 |
In this example, ``FeatureFPARMv8`` is ``SubtargetFeature`` record initialised |
| 215 |
with some values. The names of the classes are defined via the |
| 216 |
keyword `class` either in the same file or some other included. Most target |
| 217 |
TableGen files include the generic ones in ``include/llvm/Target``. |
| 218 |
|
| 219 |
**TableGen classes** are abstract records that are used to build and describe |
| 220 |
other records. These classes allow the end-user to build abstractions for |
| 221 |
either the domain they are targeting (such as ``Register``, ``RegisterClass``, and |
| 222 |
``Instruction`` in the LLVM code generator) or for the implementor to help factor |
| 223 |
out common properties of records (such as ``FPInst``, which is used to represent |
| 224 |
floating point instructions in the X86 backend). TableGen keeps track of all of |
| 225 |
the classes that are used to build up a definition, so the backend can find all |
| 226 |
definitions of a particular class, such as ``Instruction``. |
| 227 |
|
| 228 |
.. code-block:: text |
| 229 |
|
| 230 |
class ProcNoItin<string Name, list<SubtargetFeature> Features> |
| 231 |
: Processor<Name, NoItineraries, Features>; |
| 232 |
|
| 233 |
Here, the class ``ProcNoItin``, receiving parameters ``Name`` of type ``string`` and |
| 234 |
a list of target features is specializing the class ``Processor`` by passing the |
| 235 |
arguments down as well as hard-coding ``NoItineraries``. |
| 236 |
|
| 237 |
**TableGen multiclasses** are groups of abstract records that are instantiated |
| 238 |
all at once. Each instantiation can result in multiple TableGen definitions. |
| 239 |
If a multiclass inherits from another multiclass, the definitions in the |
| 240 |
sub-multiclass become part of the current multiclass, as if they were declared |
| 241 |
in the current multiclass. |
| 242 |
|
| 243 |
.. code-block:: text |
| 244 |
|
| 245 |
multiclass ro_signed_pats<string T, string Rm, dag Base, dag Offset, dag Extend, |
| 246 |
dag address, ValueType sty> { |
| 247 |
def : Pat<(i32 (!cast<SDNode>("sextload" # sty) address)), |
| 248 |
(!cast<Instruction>("LDRS" # T # "w_" # Rm # "_RegOffset") |
| 249 |
Base, Offset, Extend)>; |
| 250 |
|
| 251 |
def : Pat<(i64 (!cast<SDNode>("sextload" # sty) address)), |
| 252 |
(!cast<Instruction>("LDRS" # T # "x_" # Rm # "_RegOffset") |
| 253 |
Base, Offset, Extend)>; |
| 254 |
} |
| 255 |
|
| 256 |
defm : ro_signed_pats<"B", Rm, Base, Offset, Extend, |
| 257 |
!foreach(decls.pattern, address, |
| 258 |
!subst(SHIFT, imm_eq0, decls.pattern)), |
| 259 |
i8>; |
| 260 |
|
| 261 |
See the :doc:`TableGen Programmer's Reference <./ProgRef>` for an in-depth |
| 262 |
description of TableGen. |
| 263 |
|
| 264 |
|
| 265 |
.. _backend: |
| 266 |
.. _backends: |
| 267 |
|
| 268 |
TableGen backends |
| 269 |
================= |
| 270 |
|
| 271 |
TableGen files have no real meaning without a backend. The default operation |
| 272 |
when running ``*-tblgen`` is to print the information in a textual format, but |
| 273 |
that's only useful for debugging the TableGen files themselves. The power |
| 274 |
in TableGen is, however, to interpret the source files into an internal |
| 275 |
representation that can be generated into anything you want. |
| 276 |
|
| 277 |
Current usage of TableGen is to create huge include files with tables that you |
| 278 |
can either include directly (if the output is in the language you're coding), |
| 279 |
or be used in pre-processing via macros surrounding the include of the file. |
| 280 |
|
| 281 |
Direct output can be used if the backend already prints a table in C format |
| 282 |
or if the output is just a list of strings (for error and warning messages). |
| 283 |
Pre-processed output should be used if the same information needs to be used |
| 284 |
in different contexts (like Instruction names), so your backend should print |
| 285 |
a meta-information list that can be shaped into different compile-time formats. |
| 286 |
|
| 287 |
See :doc:`TableGen BackEnds <./BackEnds>` for a list of available |
| 288 |
backends, and see the :doc:`TableGen Backend Developer's Guide <./BackGuide>` |
| 289 |
for information on how to write and debug a new backend. |
| 290 |
|
| 291 |
Tools and Resources |
| 292 |
=================== |
| 293 |
|
| 294 |
In addition to this documentation, a list of tools and resources for TableGen |
| 295 |
can be found in TableGen's |
| 296 |
`README <https://github.com/llvm/llvm-project/blob/main/llvm/utils/TableGen/README.md>`_. |
| 297 |
|
| 298 |
TableGen Deficiencies |
| 299 |
===================== |
| 300 |
|
| 301 |
Despite being very generic, TableGen has some deficiencies that have been |
| 302 |
pointed out numerous times. The common theme is that, while TableGen allows |
| 303 |
you to build domain-specific languages, the final languages that you create |
| 304 |
lack the power of other DSLs, which in turn considerably increases the size |
| 305 |
and complexity of TableGen files. |
| 306 |
|
| 307 |
At the same time, TableGen allows you to create virtually any meaning of |
| 308 |
the basic concepts via custom-made backends, which can pervert the original |
| 309 |
design and make it very hard for newcomers to understand the evil TableGen |
| 310 |
file. |
| 311 |
|
| 312 |
There are some in favor of extending the semantics even more, but making sure |
| 313 |
backends adhere to strict rules. Others suggest moving to fewer, |
| 314 |
more powerful DSLs designed with specific purposes, or even reusing existing |
| 315 |
DSLs. |