| 1 |
Protocol Buffers - Google's data interchange format |
| 2 |
=================================================== |
| 3 |
|
| 4 |
Copyright 2008 Google Inc. |
| 5 |
|
| 6 |
This directory contains conformance tests for testing completeness and |
| 7 |
correctness of Protocol Buffers implementations. These tests are designed |
| 8 |
to be easy to run against any Protocol Buffers implementation. |
| 9 |
|
| 10 |
This directory contains the tester process `conformance-test`, which |
| 11 |
contains all of the tests themselves. Then separate programs written |
| 12 |
in whatever language you want to test communicate with the tester |
| 13 |
program over a pipe. |
| 14 |
|
| 15 |
If you're not using Bazel to run these tests, make sure you build the C++ |
| 16 |
tester code beforehand, e.g. from the base directory: |
| 17 |
|
| 18 |
$ cmake . -Dprotobuf_BUILD_CONFORMANCE=ON && cmake --build . |
| 19 |
|
| 20 |
This will produce a `conformance_test_runner` binary that can be used to run |
| 21 |
conformance tests on any executable. Pass it `--help` for more information. |
| 22 |
|
| 23 |
Running the tests for C++ |
| 24 |
------------------------- |
| 25 |
|
| 26 |
To run the tests against the C++ implementation, run: |
| 27 |
|
| 28 |
$ bazel test //src:conformance_test |
| 29 |
|
| 30 |
Or alternatively with CMake: |
| 31 |
|
| 32 |
$ ctest -R conformance_cpp_test |
| 33 |
|
| 34 |
Running the tests for other languages |
| 35 |
------------------------------------- |
| 36 |
|
| 37 |
All of the languages in the Protobuf source tree are set up to run conformance |
| 38 |
tests using similar patterns. You can either use Bazel to run the |
| 39 |
`conformance_test` target defined in the language's root `BUILD.bazel` file, |
| 40 |
or create an executable for a custom test and pass it to |
| 41 |
`conformance_test_runner`. |
| 42 |
|
| 43 |
Note: CMake can be used to build the conformance test runner, but not any of |
| 44 |
the conformance test executables outside C++. So if you aren't using Bazel |
| 45 |
you'll need to create the executable you pass to `conformance_test_runner` via |
| 46 |
some alternate build system. |
| 47 |
|
| 48 |
While we plan to model all our supported languages more completely in Bazel, |
| 49 |
today some of them are a bit tricky to run. Below is a list of the commands |
| 50 |
(and prerequisites) to run each language's conformance tests. |
| 51 |
|
| 52 |
Java: |
| 53 |
|
| 54 |
$ bazel test //java/core:conformance_test //java/lite:conformance_test |
| 55 |
|
| 56 |
Python: |
| 57 |
|
| 58 |
$ bazel test //python:conformance_test |
| 59 |
|
| 60 |
Python C++: |
| 61 |
|
| 62 |
$ bazel test //python:conformance_test_cpp --define=use_fast_cpp_protos=true |
| 63 |
|
| 64 |
C#: |
| 65 |
|
| 66 |
$ `which dotnet || echo "You must have dotnet installed!" |
| 67 |
$ `bazel test //csharp:conformance_test \ |
| 68 |
--action_env=DOTNET_CLI_TELEMETRY_OPTOUT=1 --test_env=DOTNET_CLI_HOME=~ \ |
| 69 |
--action_env=DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 |
| 70 |
|
| 71 |
Objective-C (Mac only): |
| 72 |
|
| 73 |
$ `bazel test //objectivec:conformance_test --macos_minimum_os=10.9 |
| 74 |
|
| 75 |
Ruby: |
| 76 |
|
| 77 |
$ [[ $(ruby --version) == "ruby"* ]] || echo "Select a C Ruby!" |
| 78 |
$ bazel test //ruby:conformance_test --define=ruby_platform=c \ |
| 79 |
--action_env=PATH --action_env=GEM_PATH --action_env=GEM_HOME |
| 80 |
|
| 81 |
JRuby: |
| 82 |
|
| 83 |
$ [[ $(ruby --version) == "jruby"* ]] || echo "Switch to Java Ruby!" |
| 84 |
$ bazel test //ruby:conformance_test_jruby --define=ruby_platform=java \ |
| 85 |
--action_env=PATH --action_env=GEM_PATH --action_env=GEM_HOME |
| 86 |
|
| 87 |
Testing other Protocol Buffer implementations |
| 88 |
--------------------------------------------- |
| 89 |
|
| 90 |
To run these tests against a new Protocol Buffers implementation, write a |
| 91 |
program in your language that uses the protobuf implementation you want |
| 92 |
to test. This program should implement the testing protocol defined in |
| 93 |
[conformance.proto](https://github.com/protocolbuffers/protobuf/blob/main/conformance/conformance.proto). |
| 94 |
This is designed to be as easy as possible: the C++ version is only |
| 95 |
150 lines and is a good example for what this program should look like |
| 96 |
(see [conformance_cpp.cc](https://github.com/protocolbuffers/protobuf/blob/main/conformance/conformance_cpp.cc)). |
| 97 |
The program only needs to be able to read from stdin and write to stdout. |
| 98 |
|
| 99 |
Portability |
| 100 |
----------- |
| 101 |
|
| 102 |
Note that the test runner currently does not work on Windows. Patches |
| 103 |
to fix this are welcome! (But please get in touch first to settle on |
| 104 |
a general implementation strategy). |