| 1 |
# GPU Math Conformance Tests |
| 2 |
|
| 3 |
## Overview |
| 4 |
|
| 5 |
This test suite provides a framework to systematically measure the accuracy of math functions on GPUs and verify their conformance with standards like OpenCL. |
| 6 |
|
| 7 |
While the primary focus is validating the implementations in the C standard math library (LLVM-libm), these tests can also be executed against other math library providers, such as CUDA Math and HIP Math, for comparison. |
| 8 |
|
| 9 |
The goals of this project are to empower LLVM-libm contributors with a robust tool for validating their implementations and to build trust with end-users by providing transparent accuracy data. |
| 10 |
|
| 11 |
### Table of Contents |
| 12 |
|
| 13 |
- [Getting Started](#getting-started) |
| 14 |
- [Running the Tests](#running-the-tests) |
| 15 |
- [Adding New Tests](#adding-new-tests) |
| 16 |
|
| 17 |
## Getting Started |
| 18 |
|
| 19 |
This guide covers how to build the necessary dependencies, which include the new Offload API and the C standard library for both host and GPU targets. |
| 20 |
|
| 21 |
### System Requirements |
| 22 |
|
| 23 |
Before you begin, ensure your system meets the following requirements: |
| 24 |
|
| 25 |
- A system with an AMD or NVIDIA GPU. |
| 26 |
- The latest proprietary GPU drivers installed. |
| 27 |
- The corresponding development SDK for your hardware: |
| 28 |
- **AMD:** [ROCm SDK](https://rocm.docs.amd.com) |
| 29 |
- **NVIDIA:** [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit) |
| 30 |
|
| 31 |
### Building the Dependencies |
| 32 |
|
| 33 |
The official documentation for building LLVM-libc for GPUs provides a detailed guide and should be considered the primary reference. Please follow the instructions in the **"Standard runtimes build"** section of that guide: |
| 34 |
|
| 35 |
- [Building the GPU C library (Official Documentation)](https://libc.llvm.org/gpu/building.html) |
| 36 |
|
| 37 |
> [!IMPORTANT] |
| 38 |
> For the conformance tests, the standard `cmake` command from the official documentation must be adapted slightly. You must also add `libc` to the main `-DLLVM_ENABLE_RUNTIMES` list. This is a crucial step because the tests need a host-side build of `libc` to use as the reference oracle for validating GPU results. |
| 39 |
|
| 40 |
## Running the Tests |
| 41 |
|
| 42 |
### Default Test |
| 43 |
|
| 44 |
To build and run the conformance test for a given function (e.g., `logf`) against the default C standard math library `llvm-libm` provider, use the following command. This will execute the test on all available and supported platforms. |
| 45 |
|
| 46 |
```bash |
| 47 |
ninja -C build/runtimes/runtimes-bins offload.conformance.logf |
| 48 |
``` |
| 49 |
|
| 50 |
### Testing Other Providers |
| 51 |
|
| 52 |
Once the test binary has been built, you can run it against other math library providers using the `--test-configs` flag. |
| 53 |
|
| 54 |
- **For `cuda-math` on an NVIDIA GPU:** |
| 55 |
|
| 56 |
```bash |
| 57 |
./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=cuda-math:cuda |
| 58 |
``` |
| 59 |
|
| 60 |
- **For `hip-math` on an AMD GPU:** |
| 61 |
|
| 62 |
```bash |
| 63 |
./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=hip-math:amdgpu |
| 64 |
``` |
| 65 |
|
| 66 |
You can also run all available configurations for a test with: |
| 67 |
|
| 68 |
```bash |
| 69 |
./build/runtimes/runtimes-bins/offload/logf.conformance --test-configs=all |
| 70 |
``` |
| 71 |
|
| 72 |
## Adding New Tests |
| 73 |
|
| 74 |
To add a conformance test for a new math function, follow these steps: |
| 75 |
|
| 76 |
1. **Implement the Device Kernels**: Create a kernel wrapper for the new function in each provider's source file. For CUDA Math and HIP Math, you must also add a forward declaration for the vendor function in `/device_code/DeviceAPIs.hpp`. |
| 77 |
|
| 78 |
2. **Implement the Host Test**: Create a new `.cpp` file in `/tests`. This file defines the `FunctionConfig` (function and kernel names, as well as ULP tolerance) and the input generation strategy. |
| 79 |
|
| 80 |
- Use **exhaustive testing** (`ExhaustiveGenerator`) for functions with small input spaces (e.g., half-precision functions and single-precision univariate functions). This strategy iterates over every representable point in the input space, ensuring complete coverage. |
| 81 |
- Use **randomized testing** (`RandomGenerator`) for functions with large input spaces (e.g., single-precision bivariate and double-precision functions), where exhaustive testing is computationally infeasible. Although not exhaustive, this strategy is deterministic, using a fixed seed to sample a large, reproducible subset of points from the input space. |
| 82 |
|
| 83 |
3. **Add the Build Target**: Add a new `add_conformance_test(...)` entry to `/tests/CMakeLists.txt` to make the test buildable. |