| 1 |
JGrep is a command line tool and API for parsing JSON documents based on logical expressions. |
| 2 |
|
| 3 |
### Installation: |
| 4 |
|
| 5 |
jgrep is available as a gem: |
| 6 |
|
| 7 |
gem install jgrep |
| 8 |
|
| 9 |
### JGrep binary usage: |
| 10 |
|
| 11 |
jgrep [expression] -i foo.json |
| 12 |
|
| 13 |
or |
| 14 |
|
| 15 |
cat "foo.json" | jgrep [expression] |
| 16 |
|
| 17 |
### Flags: |
| 18 |
|
| 19 |
-s, --simple [FIELDS] : Greps the JSON and only returns the value of the field(s) specified |
| 20 |
-c, --compat : Returns the JSON in its non-pretty flat form |
| 21 |
-n, --stream : Specify continuous input |
| 22 |
-f, --flatten : Flatten the results as much as possible |
| 23 |
-i, --input [FILENAME] : Target JSON file to use as input |
| 24 |
-q, --quiet : Quiet; don't write to stdout. Exit with zero status if match found. |
| 25 |
-v, --verbose : Verbose output that will list a document if it fails to parse |
| 26 |
--start FIELD : Starts the grep at a specific key in the document |
| 27 |
--slice [RANGE] : A range of the form 'n' or 'n..m', indicating which documents to extract from the final output |
| 28 |
|
| 29 |
### Expressions: |
| 30 |
|
| 31 |
JGrep uses the following logical symbols to define expressions. |
| 32 |
|
| 33 |
'and' : |
| 34 |
- [statement] and [statement] |
| 35 |
|
| 36 |
Evaluates to true if both statements are true |
| 37 |
|
| 38 |
'or' : |
| 39 |
- [statement] and [statement] |
| 40 |
|
| 41 |
Evaluates true if either statement is true |
| 42 |
|
| 43 |
'not' : |
| 44 |
- ! [statement] |
| 45 |
- not [statement] |
| 46 |
|
| 47 |
Inverts the value of statement |
| 48 |
|
| 49 |
'+' |
| 50 |
- +[value] |
| 51 |
|
| 52 |
Returns true if value is present in the json document |
| 53 |
|
| 54 |
'-' |
| 55 |
- -[value] |
| 56 |
|
| 57 |
Returns true if value is not present in the json doument |
| 58 |
|
| 59 |
'(' and ')' |
| 60 |
|
| 61 |
- (expression1) and expression2 |
| 62 |
|
| 63 |
Performs the operations inside the perentheses first. |
| 64 |
|
| 65 |
### Statements: |
| 66 |
|
| 67 |
A statement is defined as some value in a json document compared to another value. |
| 68 |
Available comparison operators are '=', '<', '>', '<=', '>=' |
| 69 |
|
| 70 |
Examples: |
| 71 |
|
| 72 |
foo.bar=1 |
| 73 |
foo.bar>0 |
| 74 |
foo.bar<=1.3 |
| 75 |
|
| 76 |
### Complex expressions: |
| 77 |
|
| 78 |
Given a json document, {"foo":1, "bar":null}, the following are examples of valid expressions |
| 79 |
|
| 80 |
Examples: |
| 81 |
|
| 82 |
+foo |
| 83 |
|
| 84 |
... returns true |
| 85 |
|
| 86 |
-bar |
| 87 |
|
| 88 |
... returns false |
| 89 |
|
| 90 |
+foo and !(foo=2) |
| 91 |
|
| 92 |
... returns true |
| 93 |
|
| 94 |
!(foo>=2 and bar=null) or !(bar=null) |
| 95 |
|
| 96 |
... returns true |
| 97 |
|
| 98 |
### CLI missing an expression: |
| 99 |
|
| 100 |
If JGrep is executed without a set expression, it will return an unmodified JSON document. The |
| 101 |
-s flag can still be applied to the result. |
| 102 |
|
| 103 |
### In document comparison: |
| 104 |
|
| 105 |
If a document contains an array, the '[' and ']' operators can be used to define a comparison where |
| 106 |
statements are checked for truth on a per element basis which will then be combined. |
| 107 |
|
| 108 |
Example: |
| 109 |
|
| 110 |
[foo.bar1=1 and foo.bar2=2] |
| 111 |
|
| 112 |
on |
| 113 |
|
| 114 |
[ |
| 115 |
{ |
| 116 |
"foo": [ |
| 117 |
{ |
| 118 |
"bar1":1 |
| 119 |
}, |
| 120 |
{ |
| 121 |
"bar2":2 |
| 122 |
} |
| 123 |
] |
| 124 |
}, |
| 125 |
{ |
| 126 |
"foo": [ |
| 127 |
{ |
| 128 |
"bar1":0 |
| 129 |
}, |
| 130 |
{ |
| 131 |
"bar2":0 |
| 132 |
} |
| 133 |
] |
| 134 |
} |
| 135 |
] |
| 136 |
|
| 137 |
will return |
| 138 |
|
| 139 |
[ |
| 140 |
{ |
| 141 |
"foo": [ |
| 142 |
{ |
| 143 |
"bar1": 1 |
| 144 |
}, |
| 145 |
{ |
| 146 |
"bar2": 2 |
| 147 |
} |
| 148 |
] |
| 149 |
} |
| 150 |
] |
| 151 |
|
| 152 |
|
| 153 |
**Note**: In document comparison cannot be nested. |
| 154 |
|
| 155 |
### The -s flag: |
| 156 |
|
| 157 |
The s flag simplifies the output returned by JGrep. Given a JSON document |
| 158 |
|
| 159 |
[{"a":1, "b":2, "c":3}, {"a":3, "b":2, "c":1}] |
| 160 |
|
| 161 |
a JGrep invocation like |
| 162 |
|
| 163 |
cat my.json | jgrep "a=1" -s b |
| 164 |
|
| 165 |
will output |
| 166 |
|
| 167 |
1 |
| 168 |
|
| 169 |
The s flag can also be used with multiple field, which will return JSON as output which only contain the specified fields. |
| 170 |
**Note**: Separate fields by a space and enclose all fields in quotes (see example below) |
| 171 |
|
| 172 |
Given: |
| 173 |
|
| 174 |
[{"a":1, "b":2, "c":3}, {"a":3, "b":2, "c":1}] |
| 175 |
|
| 176 |
a JGrep invocation like |
| 177 |
|
| 178 |
cat my.json | jgrep "a>0" -s "a c" |
| 179 |
|
| 180 |
will output |
| 181 |
|
| 182 |
[ |
| 183 |
{ |
| 184 |
"a" : 1, |
| 185 |
"c" : 3 |
| 186 |
}, |
| 187 |
{ |
| 188 |
"a" : 3, |
| 189 |
"c" : 1 |
| 190 |
} |
| 191 |
] |
| 192 |
|
| 193 |
### The --start flag: |
| 194 |
|
| 195 |
Some documents do not comply to our expected format, they might have an array embedded deep in a field. The --start |
| 196 |
flag lets you pick a starting point for the grep. |
| 197 |
|
| 198 |
An example document can be seen here: |
| 199 |
|
| 200 |
{"results": [ |
| 201 |
{"name":"Jack", "surname":"Smith"}, |
| 202 |
{"name":"Jill", "surname":"Jones"} |
| 203 |
] |
| 204 |
} |
| 205 |
|
| 206 |
This document does not comply to our standard but does contain data that can be searched - the _results_ field. |
| 207 |
We can use the --start flat to tell jgrep to start looking for data in that field: |
| 208 |
|
| 209 |
<pre> |
| 210 |
$ cat my.json | jgrep --start results name=Jack -s surname |
| 211 |
Smith |
| 212 |
</pre> |
| 213 |
|
| 214 |
### The --slice flag |
| 215 |
|
| 216 |
Allows the user to provide an int or range to slice an array of |
| 217 |
results with, in particular so a single element can be extracted, e.g. |
| 218 |
|
| 219 |
$ echo '[{"foo": {"bar": "baz"}}, {"foo": {"bar":"baz"}}]' | |
| 220 |
jgrep "foo.bar=baz" --slice 0 |
| 221 |
{ |
| 222 |
"foo": { |
| 223 |
"bar": "baz" |
| 224 |
} |
| 225 |
} |
| 226 |
|
| 227 |
### The --stream flag |
| 228 |
|
| 229 |
With the --stream or -n flag, jgrep will process multiple JSON inputs (newline |
| 230 |
separated) until standard input is closed. Each JSON input will be processed |
| 231 |
as usual, but the output immediately printed. |
| 232 |
|
| 233 |
### JGrep Gem usage: |
| 234 |
|
| 235 |
require 'jgrep' |
| 236 |
|
| 237 |
json = File.read("yourfile.json") |
| 238 |
expression = "foo=1 or bar=1" |
| 239 |
|
| 240 |
JGrep::jgrep(json, expression) |
| 241 |
|
| 242 |
sflags = "foo" |
| 243 |
|
| 244 |
JGrep::jgrep(json, expression, sflags) |
| 245 |
|