Debian Codegraph @

readme.markdown

raw
1 # JSONStream
2
3 streaming JSON.parse and stringify
4
5 ![](https://secure.travis-ci.org/dominictarr/JSONStream.png?branch=master)
6
7 ## install
8 ```npm install JSONStream```
9
10 ## example
11
12 ``` js
13
14 var request = require('request')
15 , JSONStream = require('JSONStream')
16 , es = require('event-stream')
17
18 request({url: 'http://isaacs.couchone.com/registry/_all_docs'})
19 .pipe(JSONStream.parse('rows.*'))
20 .pipe(es.mapSync(function (data) {
21 console.error(data)
22 return data
23 }))
24 ```
25
26 ## JSONStream.parse(path)
27
28 parse stream of values that match a path
29
30 ``` js
31 JSONStream.parse('rows.*.doc')
32 ```
33
34 The `..` operator is the recursive descent operator from [JSONPath](http://goessner.net/articles/JsonPath/), which will match a child at any depth (see examples below).
35
36 If your keys have keys that include `.` or `*` etc, use an array instead.
37 `['row', true, /^doc/]`.
38
39 If you use an array, `RegExp`s, booleans, and/or functions. The `..` operator is also available in array representation, using `{recurse: true}`.
40 any object that matches the path will be emitted as 'data' (and `pipe`d down stream)
41
42 If `path` is empty or null, no 'data' events are emitted.
43
44 If you want to have keys emitted, you can prefix your `*` operator with `$`: `obj.$*` - in this case the data passed to the stream is an object with a `key` holding the key and a `value` property holding the data.
45
46 ### Examples
47
48 query a couchdb view:
49
50 ``` bash
51 curl -sS localhost:5984/tests/_all_docs&include_docs=true
52 ```
53 you will get something like this:
54
55 ``` js
56 {"total_rows":129,"offset":0,"rows":[
57 { "id":"change1_0.6995461115147918"
58 , "key":"change1_0.6995461115147918"
59 , "value":{"rev":"1-e240bae28c7bb3667f02760f6398d508"}
60 , "doc":{
61 "_id": "change1_0.6995461115147918"
62 , "_rev": "1-e240bae28c7bb3667f02760f6398d508","hello":1}
63 },
64 { "id":"change2_0.6995461115147918"
65 , "key":"change2_0.6995461115147918"
66 , "value":{"rev":"1-13677d36b98c0c075145bb8975105153"}
67 , "doc":{
68 "_id":"change2_0.6995461115147918"
69 , "_rev":"1-13677d36b98c0c075145bb8975105153"
70 , "hello":2
71 }
72 },
73 ]}
74
75 ```
76
77 we are probably most interested in the `rows.*.doc`
78
79 create a `Stream` that parses the documents from the feed like this:
80
81 ``` js
82 var stream = JSONStream.parse(['rows', true, 'doc']) //rows, ANYTHING, doc
83
84 stream.on('data', function(data) {
85 console.log('received:', data);
86 });
87 //emits anything from _before_ the first match
88 stream.on('header', function (data) {
89 console.log('header:', data) // => {"total_rows":129,"offset":0}
90 })
91
92 ```
93 awesome!
94
95 In case you wanted the contents the doc emitted:
96
97 ``` js
98 var stream = JSONStream.parse(['rows', true, 'doc', {emitKey: true}]) //rows, ANYTHING, doc, items in docs with keys
99
100 stream.on('data', function(data) {
101 console.log('key:', data.key);
102 console.log('value:', data.value);
103 });
104
105 ```
106
107 You can also emit the path:
108
109 ``` js
110 var stream = JSONStream.parse(['rows', true, 'doc', {emitPath: true}]) //rows, ANYTHING, doc, items in docs with keys
111
112 stream.on('data', function(data) {
113 console.log('path:', data.path);
114 console.log('value:', data.value);
115 });
116
117 ```
118
119 ### recursive patterns (..)
120
121 `JSONStream.parse('docs..value')`
122 (or `JSONStream.parse(['docs', {recurse: true}, 'value'])` using an array)
123 will emit every `value` object that is a child, grand-child, etc. of the
124 `docs` object. In this example, it will match exactly 5 times at various depth
125 levels, emitting 0, 1, 2, 3 and 4 as results.
126
127 ```js
128 {
129 "total": 5,
130 "docs": [
131 {
132 "key": {
133 "value": 0,
134 "some": "property"
135 }
136 },
137 {"value": 1},
138 {"value": 2},
139 {"blbl": [{}, {"a":0, "b":1, "value":3}, 10]},
140 {"value": 4}
141 ]
142 }
143 ```
144
145 ## JSONStream.parse(pattern, map)
146
147 provide a function that can be used to map or filter
148 the json output. `map` is passed the value at that node of the pattern,
149 if `map` return non-nullish (anything but `null` or `undefined`)
150 that value will be emitted in the stream. If it returns a nullish value,
151 nothing will be emitted.
152
153 `JSONStream` also emits `'header'` and `'footer'` events,
154 the `'header'` event contains anything in the output that was before
155 the first match, and the `'footer'`, is anything after the last match.
156
157 ## JSONStream.stringify(open, sep, close)
158
159 Create a writable stream.
160
161 you may pass in custom `open`, `close`, and `seperator` strings.
162 But, by default, `JSONStream.stringify()` will create an array,
163 (with default options `open='[\n', sep='\n,\n', close='\n]\n'`)
164
165 If you call `JSONStream.stringify(false)`
166 the elements will only be seperated by a newline.
167
168 If you only write one item this will be valid JSON.
169
170 If you write many items,
171 you can use a `RegExp` to split it into valid chunks.
172
173 ## JSONStream.stringifyObject(open, sep, close)
174
175 Very much like `JSONStream.stringify`,
176 but creates a writable stream for objects instead of arrays.
177
178 Accordingly, `open='{\n', sep='\n,\n', close='\n}\n'`.
179
180 When you `.write()` to the stream you must supply an array with `[ key, data ]`
181 as the first argument.
182
183 ## unix tool
184
185 query npm to see all the modules that browserify has ever depended on.
186
187 ``` bash
188 curl https://registry.npmjs.org/browserify | JSONStream 'versions.*.dependencies'
189 ```
190
191 ## numbers
192
193 numbers will be emitted as numbers.
194 huge numbers that cannot be represented in memory as javascript numbers will be emitted as strings.
195 cf https://github.com/creationix/jsonparse/commit/044b268f01c4b8f97fb936fc85d3bcfba179e5bb for details.
196
197 ## Acknowlegements
198
199 this module depends on https://github.com/creationix/jsonparse
200 by Tim Caswell
201 and also thanks to Florent Jaby for teaching me about parsing with:
202 https://github.com/Floby/node-json-streams
203
204 ## license
205
206 Dual-licensed under the MIT License or the Apache License, version 2.0
207