Debian Codegraph @

offload/unittests/Conformance/README.md

raw
1 # GPU Math Conformance Tests
2
3 ## Overview
4
5 This test suite provides a framework to systematically measure the accuracy of math functions on GPUs and verify their conformance with standards like OpenCL.
6
7 While the primary focus is validating the implementations in the C standard math library (LLVM-libm), these tests can also be executed against other math library providers, such as CUDA Math and HIP Math, for comparison.
8
9 The goals of this project are to empower LLVM-libm contributors with a robust tool for validating their implementations and to build trust with end-users by providing transparent accuracy data.
10
11 ### Table of Contents
12
13 - [Getting Started](#getting-started)
14 - [Running the Tests](#running-the-tests)
15 - [Adding New Tests](#adding-new-tests)
16
17 ## Getting Started
18
19 This guide covers how to build the necessary dependencies, which include the new Offload API and the C standard library for both host and GPU targets.
20
21 ### System Requirements
22
23 Before you begin, ensure your system meets the following requirements:
24
25 - A system with an AMD or NVIDIA GPU.
26 - The latest proprietary GPU drivers installed.
27 - The corresponding development SDK for your hardware:
28 - **AMD:** [ROCm SDK](https://rocm.docs.amd.com)
29 - **NVIDIA:** [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit)
30
31 ### Building the Dependencies
32
33 The official documentation for building LLVM-libc for GPUs provides a detailed guide and should be considered the primary reference. Please follow the instructions in the **"Standard runtimes build"** section of that guide:
34
35 - [Building the GPU C library (Official Documentation)](https://libc.llvm.org/gpu/building.html)
36
37 > [!IMPORTANT]
38 > For the conformance tests, the standard `cmake` command from the official documentation must be adapted slightly. You must also add `libc` to the main `-DLLVM_ENABLE_RUNTIMES` list. This is a crucial step because the tests need a host-side build of `libc` to use as the reference oracle for validating GPU results.
39
40 ## Running the Tests
41
42 ### Default Test
43
44 To build and run the conformance test for a given function (e.g., `logf`) against the default C standard math library `llvm-libm` provider, use the following command. This will execute the test on all available and supported platforms.
45
46 ```bash
47 ninja -C build/runtimes/runtimes-bins offload.conformance.logf
48 ```
49
50 ### Testing Other Providers
51
52 Once the test binary has been built, you can run it against other math library providers using the `--test-configs` flag.
53
54 - **For `cuda-math` on an NVIDIA GPU:**
55
56 ```bash
57 ./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=cuda-math:cuda
58 ```
59
60 - **For `hip-math` on an AMD GPU:**
61
62 ```bash
63 ./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=hip-math:amdgpu
64 ```
65
66 You can also run all available configurations for a test with:
67
68 ```bash
69 ./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=all
70 ```
71
72 ## Adding New Tests
73
74 To add a conformance test for a new math function, follow these steps:
75
76 1. **Implement the Device Kernels**: Create a kernel wrapper for the new function in each provider's source file. For CUDA Math and HIP Math, you must also add a forward declaration for the vendor function in `/device_code/DeviceAPIs.hpp`.
77
78 2. **Implement the Host Test**: Create a new `.cpp` file in `/tests`. This file defines the `FunctionConfig` (function and kernel names, as well as ULP tolerance) and the input generation strategy.
79
80 - Use **exhaustive testing** (`ExhaustiveGenerator`) for functions with small input spaces (e.g., half-precision functions and single-precision univariate functions). This strategy iterates over every representable point in the input space, ensuring complete coverage.
81 - Use **randomized testing** (`RandomGenerator`) for functions with large input spaces (e.g., single-precision bivariate and double-precision functions), where exhaustive testing is computationally infeasible. Although not exhaustive, this strategy is deterministic, using a fixed seed to sample a large, reproducible subset of points from the input space.
82
83 3. **Add the Build Target**: Add a new `add_conformance_test(...)` entry to `/tests/CMakeLists.txt` to make the test buildable.