Debian Codegraph @

test/json/README.md

raw
1 The files in this subdirectory are used to help measure the performance
2 of the SQLite JSON functions, especially in relation to handling large
3 JSON inputs.
4
5 # 1.0 Prerequisites
6
7 * Standard SQLite build environment (SQLite source tree, compiler, make, etc.)
8
9 * Valgrind
10
11 * Fossil (only the "fossil xdiff" command is used by this procedure)
12
13 * tclsh
14
15 # 2.0 Setup
16
17 * Run: "`tclsh json-generator.tcl | sqlite3 json100mb.db`" to create
18 the 100 megabyte test database. Do this so that the "json100mb.db"
19 file lands in the directory from which you will run tests, not in
20 the test/json subdirectory of the source tree.
21
22 * Make a copy of "json100mb.db" into "jsonb100mb.db" - change the prefix
23 from "json" to "jsonb".
24
25 * Bring up jsonb100mb.db in the sqlite3 command-line shell.
26 Convert all of the content into JSONB using a commands like this:
27
28 > UPDATE data1 SET x=jsonb(x);
29 > VACUUM;
30
31 * Build the baseline sqlite3.c file with sqlite3.h and shell.c.
32
33 > make clean sqlite3.c
34
35 * Run "`sh json-speed-check.sh trunk`". This creates the baseline
36 profile in "jout-trunk.txt" for the preformance test using text JSON.
37
38 * Run "`sh json-speed-check.sh trunk --jsonb`". This creates the
39 baseline profile in "joutb-trunk.txt" for the performance test
40 for processing JSONB
41
42 * (Optional) Verify that the json100mb.db database really does contain
43 approximately 100MB of JSON content by running:
44
45 > SELECT sum(length(x)) FROM data1;
46 > SELECT * FROM data1 WHERE NOT json_valid(x);
47
48 # 3.0 Testing
49
50 * Build the sqlite3.c (with sqlite3.h and shell.c) to be tested.
51
52 * Run "`sh json-speed-check.sh x1`". The profile output will appear
53 in jout-x1.txt. Substitute any label you want in place of "x1".
54
55 * Run "`sh json-speed-check.sh x1 --jsonb`". The profile output will appear
56 in joutb-x1.txt. Substitute any label you want in place of "x1".
57
58 * Run the script shown below in the CLI.
59 Divide 2500 by the real elapse time from this test
60 to get an estimate for number of MB/s that the JSON parser is
61 able to process.
62
63 > .open json100mb.db
64 > .timer on
65 > WITH RECURSIVE c(n) AS (VALUES(1) UNION ALL SELECT n+1 FROM c WHERE n<25)
66 > SELECT sum(json_valid(x)) FROM c, data1;