| 1 |
The files in this subdirectory are used to help measure the performance |
| 2 |
of the SQLite JSON functions, especially in relation to handling large |
| 3 |
JSON inputs. |
| 4 |
|
| 5 |
# 1.0 Prerequisites |
| 6 |
|
| 7 |
* Standard SQLite build environment (SQLite source tree, compiler, make, etc.) |
| 8 |
|
| 9 |
* Valgrind |
| 10 |
|
| 11 |
* Fossil (only the "fossil xdiff" command is used by this procedure) |
| 12 |
|
| 13 |
* tclsh |
| 14 |
|
| 15 |
# 2.0 Setup |
| 16 |
|
| 17 |
* Run: "`tclsh json-generator.tcl | sqlite3 json100mb.db`" to create |
| 18 |
the 100 megabyte test database. Do this so that the "json100mb.db" |
| 19 |
file lands in the directory from which you will run tests, not in |
| 20 |
the test/json subdirectory of the source tree. |
| 21 |
|
| 22 |
* Make a copy of "json100mb.db" into "jsonb100mb.db" - change the prefix |
| 23 |
from "json" to "jsonb". |
| 24 |
|
| 25 |
* Bring up jsonb100mb.db in the sqlite3 command-line shell. |
| 26 |
Convert all of the content into JSONB using a commands like this: |
| 27 |
|
| 28 |
> UPDATE data1 SET x=jsonb(x); |
| 29 |
> VACUUM; |
| 30 |
|
| 31 |
* Build the baseline sqlite3.c file with sqlite3.h and shell.c. |
| 32 |
|
| 33 |
> make clean sqlite3.c |
| 34 |
|
| 35 |
* Run "`sh json-speed-check.sh trunk`". This creates the baseline |
| 36 |
profile in "jout-trunk.txt" for the preformance test using text JSON. |
| 37 |
|
| 38 |
* Run "`sh json-speed-check.sh trunk --jsonb`". This creates the |
| 39 |
baseline profile in "joutb-trunk.txt" for the performance test |
| 40 |
for processing JSONB |
| 41 |
|
| 42 |
* (Optional) Verify that the json100mb.db database really does contain |
| 43 |
approximately 100MB of JSON content by running: |
| 44 |
|
| 45 |
> SELECT sum(length(x)) FROM data1; |
| 46 |
> SELECT * FROM data1 WHERE NOT json_valid(x); |
| 47 |
|
| 48 |
# 3.0 Testing |
| 49 |
|
| 50 |
* Build the sqlite3.c (with sqlite3.h and shell.c) to be tested. |
| 51 |
|
| 52 |
* Run "`sh json-speed-check.sh x1`". The profile output will appear |
| 53 |
in jout-x1.txt. Substitute any label you want in place of "x1". |
| 54 |
|
| 55 |
* Run "`sh json-speed-check.sh x1 --jsonb`". The profile output will appear |
| 56 |
in joutb-x1.txt. Substitute any label you want in place of "x1". |
| 57 |
|
| 58 |
* Run the script shown below in the CLI. |
| 59 |
Divide 2500 by the real elapse time from this test |
| 60 |
to get an estimate for number of MB/s that the JSON parser is |
| 61 |
able to process. |
| 62 |
|
| 63 |
> .open json100mb.db |
| 64 |
> .timer on |
| 65 |
> WITH RECURSIVE c(n) AS (VALUES(1) UNION ALL SELECT n+1 FROM c WHERE n<25) |
| 66 |
> SELECT sum(json_valid(x)) FROM c, data1; |