Debian Codegraph @

third_party/spirv-cross/README.md

raw
1 # SPIRV-Cross
2
3 SPIRV-Cross is a tool designed for parsing and converting SPIR-V to other shader languages.
4
5 [![Build Status](https://travis-ci.org/KhronosGroup/SPIRV-Cross.svg?branch=master)](https://travis-ci.org/KhronosGroup/SPIRV-Cross)
6 [![Build Status](https://ci.appveyor.com/api/projects/status/github/KhronosGroup/SPIRV-Cross?svg=true&branch=master)](https://ci.appveyor.com/project/HansKristian-Work/SPIRV-Cross)
7
8 ## Features
9
10 - Convert SPIR-V to readable, usable and efficient GLSL
11 - Convert SPIR-V to readable, usable and efficient Metal Shading Language (MSL)
12 - Convert SPIR-V to readable, usable and efficient HLSL
13 - Convert SPIR-V to debuggable C++ [DEPRECATED]
14 - Convert SPIR-V to a JSON reflection format [EXPERIMENTAL]
15 - Reflection API to simplify the creation of Vulkan pipeline layouts
16 - Reflection API to modify and tweak OpDecorations
17 - Supports "all" of vertex, fragment, tessellation, geometry and compute shaders.
18
19 SPIRV-Cross tries hard to emit readable and clean output from the SPIR-V.
20 The goal is to emit GLSL or MSL that looks like it was written by a human and not awkward IR/assembly-like code.
21
22 NOTE: Individual features are expected to be mostly complete, but it is possible that certain obscure GLSL features are not yet supported.
23 However, most missing features are expected to be "trivial" improvements at this stage.
24
25 ## Building
26
27 SPIRV-Cross has been tested on Linux, iOS/OSX, Windows and Android. CMake is the main build system.
28
29 ### Linux and macOS
30
31 Building with CMake is recommended, as it is the only build system which is tested in continuous integration.
32 It is also the only build system which has install commands and other useful build system features.
33
34 However, you can just run `make` on the command line as a fallback if you only care about the CLI tool.
35
36 A non-ancient GCC (4.8+) or Clang (3.x+) compiler is required as SPIRV-Cross uses C++11 extensively.
37
38 ### Windows
39
40 Building with CMake is recommended, which is the only way to target MSVC.
41 MinGW-w64 based compilation works with `make` as a fallback.
42
43 ### Android
44
45 SPIRV-Cross is only useful as a library here. Use the CMake build to link SPIRV-Cross to your project.
46
47 ### C++ exceptions
48
49 The make and CMake build flavors offer the option to treat exceptions as assertions. To disable exceptions for make just append `SPIRV_CROSS_EXCEPTIONS_TO_ASSERTIONS=1` to the command line. For CMake append `-DSPIRV_CROSS_EXCEPTIONS_TO_ASSERTIONS=ON`. By default exceptions are enabled.
50
51 ### Static, shared and CLI
52
53 You can use `-DSPIRV_CROSS_STATIC=ON/OFF` `-DSPIRV_CROSS_SHARED=ON/OFF` `-DSPIRV_CROSS_CLI=ON/OFF` to control which modules are built (and installed).
54
55 ## Usage
56
57 ### Using the C++ API
58
59 The C++ API is the main API for SPIRV-Cross. For more in-depth documentation than what's provided in this README,
60 please have a look at the [Wiki](https://github.com/KhronosGroup/SPIRV-Cross/wiki).
61 **NOTE**: This API is not guaranteed to be ABI-stable, and it is highly recommended to link against this API statically.
62 The API is generally quite stable, but it can change over time, see the C API for more stability.
63
64 To perform reflection and convert to other shader languages you can use the SPIRV-Cross API.
65 For example:
66
67 ```c++
68 #include "spirv_glsl.hpp"
69 #include <vector>
70 #include <utility>
71
72 extern std::vector<uint32_t> load_spirv_file();
73
74 int main()
75 {
76 // Read SPIR-V from disk or similar.
77 std::vector<uint32_t> spirv_binary = load_spirv_file();
78
79 spirv_cross::CompilerGLSL glsl(std::move(spirv_binary));
80
81 // The SPIR-V is now parsed, and we can perform reflection on it.
82 spirv_cross::ShaderResources resources = glsl.get_shader_resources();
83
84 // Get all sampled images in the shader.
85 for (auto &resource : resources.sampled_images)
86 {
87 unsigned set = glsl.get_decoration(resource.id, spv::DecorationDescriptorSet);
88 unsigned binding = glsl.get_decoration(resource.id, spv::DecorationBinding);
89 printf("Image %s at set = %u, binding = %u\n", resource.name.c_str(), set, binding);
90
91 // Modify the decoration to prepare it for GLSL.
92 glsl.unset_decoration(resource.id, spv::DecorationDescriptorSet);
93
94 // Some arbitrary remapping if we want.
95 glsl.set_decoration(resource.id, spv::DecorationBinding, set * 16 + binding);
96 }
97
98 // Set some options.
99 spirv_cross::CompilerGLSL::Options options;
100 options.version = 310;
101 options.es = true;
102 glsl.set_options(options);
103
104 // Compile to GLSL, ready to give to GL driver.
105 std::string source = glsl.compile();
106 }
107 ```
108
109 ### Using the C API wrapper
110
111 To facilitate C compatibility and compatibility with foreign programming languages, a C89-compatible API wrapper is provided. Unlike the C++ API,
112 the goal of this wrapper is to be fully stable, both API and ABI-wise.
113 This is the only interface which is supported when building SPIRV-Cross as a shared library.
114
115 An important point of the wrapper is that all memory allocations are contained in the `spvc_context`.
116 This simplifies the use of the API greatly. However, you should destroy the context as soon as reasonable,
117 or use `spvc_context_release_allocations()` if you intend to reuse the `spvc_context` object again soon.
118
119 Most functions return a `spvc_result`, where `SPVC_SUCCESS` is the only success code.
120 For brevity, the code below does not do any error checking.
121
122 ```c
123 #include <spirv_cross_c.h>
124
125 const SpvId *spirv = get_spirv_data();
126 size_t word_count = get_spirv_word_count();
127
128 spvc_context context = NULL;
129 spvc_parsed_ir ir = NULL;
130 spvc_compiler compiler_glsl = NULL;
131 spvc_compiler_options options = NULL;
132 spvc_resources resources = NULL;
133 const spvc_reflected_resource *list = NULL;
134 const char *result = NULL;
135 size_t count;
136 size_t i;
137
138 // Create context.
139 spvc_context_create(&context);
140
141 // Set debug callback.
142 spvc_context_set_error_callback(context, error_callback, userdata);
143
144 // Parse the SPIR-V.
145 spvc_context_parse_spirv(context, spirv, word_count, &ir);
146
147 // Hand it off to a compiler instance and give it ownership of the IR.
148 spvc_context_create_compiler(context, SPVC_BACKEND_GLSL, ir, SPVC_CAPTURE_MODE_TAKE_OWNERSHIP, &compiler_glsl);
149
150 // Do some basic reflection.
151 spvc_compiler_create_shader_resources(compiler_glsl, &resources);
152 spvc_resources_get_resource_list_for_type(resources, SPVC_RESOURCE_TYPE_UNIFORM_BUFFER, &list, &count);
153
154 for (i = 0; i < count; i++)
155 {
156 printf("ID: %u, BaseTypeID: %u, TypeID: %u, Name: %s\n", list[i].id, list[i].base_type_id, list[i].type_id,
157 list[i].name);
158 printf(" Set: %u, Binding: %u\n",
159 spvc_compiler_get_decoration(compiler_glsl, list[i].id, SpvDecorationDescriptorSet),
160 spvc_compiler_get_decoration(compiler_glsl, list[i].id, SpvDecorationBinding));
161 }
162
163 // Modify options.
164 spvc_compiler_create_compiler_options(context, &options);
165 spvc_compiler_options_set_uint(options, SPVC_COMPILER_OPTION_GLSL_VERSION, 330);
166 spvc_compiler_options_set_bool(options, SPVC_COMPILER_OPTION_GLSL_ES, SPVC_FALSE);
167 spvc_compiler_install_compiler_options(compiler_glsl, options);
168
169 spvc_compiler_compile(compiler, &result);
170 printf("Cross-compiled source: %s\n", result);
171
172 // Frees all memory we allocated so far.
173 spvc_context_destroy(context);
174 ```
175
176 ### Linking
177
178 #### CMake add_subdirectory()
179
180 This is the recommended way if you are using CMake and want to link against SPIRV-Cross statically.
181
182 #### Integrating SPIRV-Cross in a custom build system
183
184 To add SPIRV-Cross to your own codebase, just copy the source and header files from root directory
185 and build the relevant .cpp files you need. Make sure to build with C++11 support, e.g. `-std=c++11` in GCC and Clang.
186 Alternatively, the Makefile generates a libspirv-cross.a static library during build that can be linked in.
187
188 #### Linking against SPIRV-Cross as a system library
189
190 It is possible to link against SPIRV-Cross when it is installed as a system library,
191 which would be mostly relevant for Unix-like platforms.
192
193 ##### pkg-config
194
195 For Unix-based systems, a pkg-config is installed for the C API, e.g.:
196
197 ```
198 $ pkg-config spirv-cross-c-shared --libs --cflags
199 -I/usr/local/include/spirv_cross -L/usr/local/lib -lspirv-cross-c-shared
200 ```
201
202 ##### CMake
203
204 If the project is installed, it can be found with `find_package()`, e.g.:
205
206 ```
207 cmake_minimum_required(VERSION 3.5)
208 set(CMAKE_C_STANDARD 99)
209 project(Test LANGUAGES C)
210
211 find_package(spirv_cross_c_shared)
212 if (spirv_cross_c_shared_FOUND)
213 message(STATUS "Found SPIRV-Cross C API! :)")
214 else()
215 message(STATUS "Could not find SPIRV-Cross C API! :(")
216 endif()
217
218 add_executable(test test.c)
219 target_link_libraries(test spirv-cross-c-shared)
220 ```
221
222 test.c:
223 ```c
224 #include <spirv_cross_c.h>
225
226 int main(void)
227 {
228 spvc_context context;
229 spvc_context_create(&context);
230 spvc_context_destroy(context);
231 }
232 ```
233
234 ### CLI
235
236 The CLI is suitable for basic cross-compilation tasks, but it cannot support the full flexibility that the API can.
237 Some examples below.
238
239 #### Creating a SPIR-V file from GLSL with glslang
240
241 ```
242 glslangValidator -H -V -o test.spv test.frag
243 ```
244
245 #### Converting a SPIR-V file to GLSL ES
246
247 ```
248 glslangValidator -H -V -o test.spv shaders/comp/basic.comp
249 ./spirv-cross --version 310 --es test.spv
250 ```
251
252 #### Converting to desktop GLSL
253
254 ```
255 glslangValidator -H -V -o test.spv shaders/comp/basic.comp
256 ./spirv-cross --version 330 --no-es test.spv --output test.comp
257 ```
258
259 #### Disable prettifying optimizations
260
261 ```
262 glslangValidator -H -V -o test.spv shaders/comp/basic.comp
263 ./spirv-cross --version 310 --es test.spv --output test.comp --force-temporary
264 ```
265
266 ### Using shaders generated from C++ backend
267
268 Please see `samples/cpp` where some GLSL shaders are compiled to SPIR-V, decompiled to C++ and run with test data.
269 Reading through the samples should explain how to use the C++ interface.
270 A simple Makefile is included to build all shaders in the directory.
271
272 ### Implementation notes
273
274 When using SPIR-V and SPIRV-Cross as an intermediate step for cross-compiling between high level languages there are some considerations to take into account,
275 as not all features used by one high-level language are necessarily supported natively by the target shader language.
276 SPIRV-Cross aims to provide the tools needed to handle these scenarios in a clean and robust way, but some manual action is required to maintain compatibility.
277
278 #### HLSL source to GLSL
279
280 ##### HLSL entry points
281
282 When using SPIR-V shaders compiled from HLSL, there are some extra things you need to take care of.
283 First make sure that the entry point is used correctly.
284 If you forget to set the entry point correctly in glslangValidator (-e MyFancyEntryPoint),
285 you will likely encounter this error message:
286
287 ```
288 Cannot end a function before ending the current block.
289 Likely cause: If this SPIR-V was created from glslang HLSL, make sure the entry point is valid.
290 ```
291
292 ##### Vertex/Fragment interface linking
293
294 HLSL relies on semantics in order to effectively link together shader stages. In the SPIR-V generated by glslang, the transformation from HLSL to GLSL ends up looking like
295
296 ```c++
297 struct VSOutput {
298 // SV_Position is rerouted to gl_Position
299 float4 position : SV_Position;
300 float4 coord : TEXCOORD0;
301 };
302
303 VSOutput main(...) {}
304 ```
305
306 ```c++
307 struct VSOutput {
308 float4 coord;
309 }
310 layout(location = 0) out VSOutput _magicNameGeneratedByGlslang;
311 ```
312
313 While this works, be aware of the type of the struct which is used in the vertex stage and the fragment stage.
314 There may be issues if the structure type name differs in vertex stage and fragment stage.
315
316 You can make use of the reflection interface to force the name of the struct type.
317
318 ```
319 // Something like this for both vertex outputs and fragment inputs.
320 compiler.set_name(varying_resource.base_type_id, "VertexFragmentLinkage");
321 ```
322
323 Some platform may require identical variable name for both vertex outputs and fragment inputs. (for example MacOSX)
324 to rename variable base on location, please add
325 ```
326 --rename-interface-variable <in|out> <location> <new_variable_name>
327 ```
328
329 #### HLSL source to legacy GLSL/ESSL
330
331 HLSL tends to emit varying struct types to pass data between vertex and fragment.
332 This is not supported in legacy GL/GLES targets, so to support this, varying structs are flattened.
333 This is done automatically, but the API user might need to be aware that this is happening in order to support all cases.
334
335 Modern GLES code like this:
336 ```c++
337 struct Output {
338 vec4 a;
339 vec2 b;
340 };
341 out Output vout;
342 ```
343
344 Is transformed into:
345 ```c++
346 struct Output {
347 vec4 a;
348 vec2 b;
349 };
350 varying vec4 Output_a;
351 varying vec2 Output_b;
352 ```
353
354 Note that now, both the struct name and the member names will participate in the linking interface between vertex and fragment, so
355 API users might want to ensure that both the struct names and member names match so that vertex outputs and fragment inputs can link properly.
356
357
358 #### Separate image samplers (HLSL/Vulkan) for backends which do not support it (GLSL)
359
360 Another thing you need to remember is when using samplers and textures in HLSL these are separable, and not directly compatible with GLSL. If you need to use this with desktop GL/GLES, you need to call `Compiler::build_combined_image_samplers` first before calling `Compiler::compile`, or you will get an exception.
361
362 ```c++
363 // From main.cpp
364 // Builds a mapping for all combinations of images and samplers.
365 compiler->build_combined_image_samplers();
366
367 // Give the remapped combined samplers new names.
368 // Here you can also set up decorations if you want (binding = #N).
369 for (auto &remap : compiler->get_combined_image_samplers())
370 {
371 compiler->set_name(remap.combined_id, join("SPIRV_Cross_Combined", compiler->get_name(remap.image_id),
372 compiler->get_name(remap.sampler_id)));
373 }
374 ```
375
376 If your target is Vulkan GLSL, `--vulkan-semantics` will emit separate image samplers as you'd expect.
377 The command line client calls `Compiler::build_combined_image_samplers` automatically, but if you're calling the library, you'll need to do this yourself.
378
379 #### Descriptor sets (Vulkan GLSL) for backends which do not support them (HLSL/GLSL/Metal)
380
381 Descriptor sets are unique to Vulkan, so make sure that descriptor set + binding is remapped to a flat binding scheme (set always 0), so that other APIs can make sense of the bindings.
382 This can be done with `Compiler::set_decoration(id, spv::DecorationDescriptorSet)`.
383
384 #### Linking by name for targets which do not support explicit locations (legacy GLSL/ESSL)
385
386 Modern GLSL and HLSL sources (and SPIR-V) relies on explicit layout(location) qualifiers to guide the linking process between shader stages,
387 but older GLSL relies on symbol names to perform the linking. When emitting shaders with older versions, these layout statements will be removed,
388 so it is important that the API user ensures that the names of I/O variables are sanitized so that linking will work properly.
389 The reflection API can rename variables, struct types and struct members to deal with these scenarios using `Compiler::set_name` and friends.
390
391 #### Clip-space conventions
392
393 SPIRV-Cross can perform some common clip space conversions on gl_Position/SV_Position by enabling `CompilerGLSL::Options.vertex.fixup_clipspace`.
394 While this can be convenient, it is recommended to modify the projection matrices instead as that can achieve the same result.
395
396 For GLSL targets, enabling this will convert a shader which assumes `[0, w]` depth range (Vulkan / D3D / Metal) into `[-w, w]` range.
397 For MSL and HLSL targets, enabling this will convert a shader in `[-w, w]` depth range (OpenGL) to `[0, w]` depth range.
398
399 By default, the CLI will not enable `fixup_clipspace`, but in the API you might want to set an explicit value using `CompilerGLSL::set_options()`.
400
401 Y-flipping of gl_Position and similar is also supported.
402 The use of this is discouraged, because relying on vertex shader Y-flipping tends to get quite messy.
403 To enable this, set `CompilerGLSL::Options.vertex.flip_vert_y` or `--flip-vert-y` in CLI.
404
405 #### Reserved identifiers
406
407 When cross-compiling, certain identifiers are considered to be reserved by the implementation.
408 Code generated by SPIRV-Cross cannot emit these identifiers as they are reserved and used for various internal purposes,
409 and such variables will typically show up as `_RESERVED_IDENTIFIER_FIXUP_`
410 or some similar name to make it more obvious that an identifier has been renamed.
411
412 Reflection output will follow the exact name specified in the SPIR-V module. It might not be a valid identifier in the C sense,
413 as it may contain non-alphanumeric/non-underscore characters.
414
415 Reserved identifiers currently assumed by the implementation are (in pseudo-regex):
416
417 - _$digit+, e.g. `_100`, `_2`
418 - _$digit+_.+, e.g. `_100_tmp`, `_2_foobar`. `_2Bar` is **not** reserved.
419 - gl_- prefix
420 - spv- prefix
421 - SPIRV_Cross prefix
422 - Double underscores (reserved by all target languages).
423
424 Members of structs also have a reserved identifier:
425 - _m$digit+$END, e.g. `_m20` and `_m40` are reserved, but not `_m40Foobar`.
426
427 ## Contributing
428
429 Contributions to SPIRV-Cross are welcome. See Testing and Licensing sections for details.
430
431 ### Testing
432
433 SPIRV-Cross maintains a test suite of shaders with reference output of how the output looks after going through a roundtrip through
434 glslangValidator/spirv-as then back through SPIRV-Cross again.
435 The reference files are stored inside the repository in order to be able to track regressions.
436
437 All pull requests should ensure that test output does not change unexpectedly. This can be tested with:
438
439 ```
440 ./checkout_glslang_spirv_tools.sh # Checks out glslang and SPIRV-Tools at a fixed revision which matches the reference output.
441 # NOTE: Some users have reported problems cloning from git:// paths. To use https:// instead pass in
442 # $ PROTOCOL=https ./checkout_glslang_spirv_tools.sh
443 # instead.
444 ./build_glslang_spirv_tools.sh # Builds glslang and SPIRV-Tools.
445 ./test_shaders.sh # Runs over all changes and makes sure that there are no deltas compared to reference files.
446 ```
447
448 `./test_shaders.sh` currently requires a Makefile setup with GCC/Clang to be set up.
449 However, on Windows, this can be rather inconvenient if a MinGW environment is not set up.
450 To use a spirv-cross binary you built with CMake (or otherwise), you can pass in an environment variable as such:
451
452 ```
453 SPIRV_CROSS_PATH=path/to/custom/spirv-cross ./test_shaders.sh
454 ```
455
456 However, when improving SPIRV-Cross there are of course legitimate cases where reference output should change.
457 In these cases, run:
458
459 ```
460 ./update_test_shaders.sh # SPIRV_CROSS_PATH also works here.
461 ```
462
463 to update the reference files and include these changes as part of the pull request.
464 Always make sure you are running the correct version of glslangValidator as well as SPIRV-Tools when updating reference files.
465 See `checkout_glslang_spirv_tools.sh` which revisions are currently expected. The revisions change regularly.
466
467 In short, the master branch should always be able to run `./test_shaders.py shaders` and friends without failure.
468 SPIRV-Cross uses Travis CI to test all pull requests, so it is not strictly needed to perform testing yourself if you have problems running it locally.
469 A pull request which does not pass testing on Travis will not be accepted however.
470
471 When adding support for new features to SPIRV-Cross, a new shader and reference file should be added which covers usage of the new shader features in question.
472 Travis CI runs the test suite with the CMake, by running `ctest`. This is a more straight-forward alternative to `./test_shaders.sh`.
473
474 ### Licensing
475
476 Contributors of new files should add a copyright header at the top of every new source code file with their copyright
477 along with the Apache 2.0 licensing stub.
478
479 ### Formatting
480
481 SPIRV-Cross uses `clang-format` to automatically format code.
482 Please use `clang-format` with the style sheet found in `.clang-format` to automatically format code before submitting a pull request.
483
484 To make things easy, the `format_all.sh` script can be used to format all
485 source files in the library. In this directory, run the following from the
486 command line:
487
488 ./format_all.sh
489
490 ## Regression testing
491
492 In shaders/ a collection of shaders are maintained for purposes of regression testing.
493 The current reference output is contained in reference/.
494 `./test_shaders.py shaders` can be run to perform regression testing.
495
496 See `./test_shaders.py --help` for more.
497
498 ### Metal backend
499
500 To test the roundtrip path GLSL -> SPIR-V -> MSL, `--msl` can be added, e.g. `./test_shaders.py --msl shaders-msl`.
501
502 ### HLSL backend
503
504 To test the roundtrip path GLSL -> SPIR-V -> HLSL, `--hlsl` can be added, e.g. `./test_shaders.py --hlsl shaders-hlsl`.
505
506 ### Updating regression tests
507
508 When legitimate changes are found, use `--update` flag to update regression files.
509 Otherwise, `./test_shaders.py` will fail with error code.
510
511 ### Mali Offline Compiler cycle counts
512
513 To obtain a CSV of static shader cycle counts before and after going through spirv-cross, add
514 `--malisc` flag to `./test_shaders`. This requires the Mali Offline Compiler to be installed in PATH.
515