Debian Codegraph @

docs/autodocs/source/ecephys_probe.rst

raw
1 :orphan:
2
3 *****************************************************************
4 Spikes Stimulus with (NWB) Extracellular Electrophysiology Probes
5 *****************************************************************
6
7 The goal of the `ecephys_probe` inputs module is to drive a network simulation using experimental data
8 recorded using extracellular electrophysiology (ecephys) multi-channel probes. Typically, BMTK and SONATA
9 uses either SONATA Spike-Train files or csv files to create virtual cells to provide synaptic inputs to
10 the cells in our model. And while SONATA are can be converted and/or generated based on observed data, the
11 `ecephys_probe` module allows us directly incorpate in-vivo spike recordings into our in-silco simulation.
12
13 This module uses `NWB 2.0 <https://pynwb.readthedocs.io/en/stable/tutorials/index.html>`_ formated files to
14 fetch unit spike-times data and convert them into synaptic stimuli. It was mainly designed for use with
15 `Allen Institute Visual Coding <https://allensdk.readthedocs.io/en/latest/visual_coding_neuropixels.html>`_ and
16 `Visual Behavior <http://portal.brain-map.org/explore/circuits/visual-behavior-neuropixels>`_ neuropixels data.
17 But can be used with external public ecephys session data, like many of those found on
18 `DANDI <https://dandiarchive.org/>`_ provided they have units with spike-times.
19
20
21 Quick Start Guide
22 #################
23
24 For pre-built examples of network simulations that use in-vivo ecephys probe data there are a number of
25 examples in the BTMK **examples/** folder; including ones for `BioNet (NEURON) <https://github.com/AllenInstitute/bmtk/tree/develop/examples/bio_neuropixels>`_ and
26 `PointNet (NEST) <https://github.com/AllenInstitute/bmtk/tree/develop/examples/point_neuropixels>`_. Once BMTK is installed on your system just run a simulation using one
27 of the simulation JSON configs:
28
29 .. code:: bash
30
31 $ python run_bmtk.py config.simulation.x.json
32
33
34 Using the `ecephys_probe` module in your simulation is all done through the
35 `simulation simulation config <./simulators.html#configuration-files>`_ file. Using your preferred text
36 editor you can open up the simulation json configuration file, and update the **inputs** section to
37 add or modify the `ecephys_probe` modules.
38
39 For example, suppose our network includes virtual thalamic **LGN** cells that synapse onto our cortical cells,
40 and we want to use spike trains from Visual Coding Session_715093703.nwb file (See `below <INSERT_LINK>`_ on
41 tricks for downloading nwb files), then we can add the following section to our json file:
42
43 .. code:: json
44
45 {
46 "inputs": {
47 "LGN_spikes": {
48 "input_type": "spikes",
49 "module": "ecephys_probe",
50 "input_file": "./session_715093703.nwb",
51 "node_set": {"population": "LGN"},
52 "mapping": "sample_with_replacement",
53 "units": {
54 "location": "LGd"
55 },
56 "interval": [10000.0, 12000.0]
57 }
58 }
59 }
60
61
62 **Where**
63 * **LGN_spikes** is the unique arbitary name we give for this particular module block (there can be multiple input blocks in the same simulation).
64 * We set **"input":** *spikes* and **"module":** *"ecephys_probe"* to instruct BMTK (and any other SONATA compliant simulator) to create virtual spike
65 trains using NWB ECEPhys Probe data. This value cannot be changed
66 * **input_file** is the path to our download NWB file.
67 * **node_set** is used to select the sub-population of cells to use for the LGN spikes inputs, in this case use all cells in the **LGN** cell population (see `the following <https://github.com/AllenInstitute/sonata/blob/master/docs/SONATA_DEVELOPER_GUIDE.md#node-sets-file>`_ for more information on SONATA node-sets).
68 * **mapping** is the method to map the NWB units to our model nodes, in this case indicating we want to randomly assign NWB units to our network cells (with replacement in case there are fewer NWB units than cells).
69 * **units** let's us filter out the exact units in our NWB file to use, here we only want the spike trains of units where the location is from the LGd.
70 * **interval** let's us filter spike trains in the NWB file based on a given time window in ms, in this case we only want spike trains that occured between 10-12 seconds (which by default will be converted 0-2 seconds in the simulation)
71
72
73 The module also allows you to use multiple NWB files at a time, where the sampling of units in the use of our simulation will be done across all NWB input_file's
74
75 .. code:: json
76
77 {
78 "inputs": {
79 "LGN_spikes": {
80 "input_type": "spikes",
81 "module": "ecephys_probe",
82 "input_file": ["./session_715093703.nwb", "./session_733123887.nwb"],
83 "node_set": {"population": "LGN"},
84 "mapping": "sample_with_replacement",
85 "units": {
86 "location": "LGd"
87 },
88 "interval": [10000.0, 12000.0]
89 }
90 }
91
92 Or if you have multiple populations that you want to use to drive input into our model, you can just use the module multiple times:
93
94
95 .. code:: json
96
97 {
98 "inputs": {
99 "LGN_spikes": {
100 "input_type": "spikes",
101 "module": "ecephys_probe",
102 "input_file": ["./session_715093703.nwb", "./session_733123887.nwb"],
103 "node_set": {"population": "LGN"},
104 "mapping": "sample_with_replacement",
105 "units": {
106 "location": "LGd"
107 },
108 "interval": [10000.0, 12000.0]
109 },
110
111 "hippocampus_spikes": {
112 "input_type": "spikes",
113 "module": "ecephys_probe",
114 "input_file": ["./session_715093703.nwb", "./session_733123887.nwb"],
115 "node_set": {"population": "LGN"},
116 "mapping": "sample_with_replacement",
117 "units": {
118 "location": ["CA1", "CA3", "Po"]
119 },
120 "interval": [135000.0, 137000.0]
121 }
122 }
123
124 Parameters
125 ##########
126 * **input_file:** - *string* or *list[String]* - A path or list of paths to NWB file(s) contain ECEPhys units spike-time recordings.
127 * **node_set** - *string* or *dict* - A filter for node (sub)population to use when assing NWB units to spikes.
128 * **mapping** - *string* - The strategy by which NWB units are assinged to simulation nodes. Current options: units_maps, sample, sample_with_replacement.
129 * **units** - *string* or *dict* - Used to filter out which units in NWB **input_file** are to be used by module, if empty will use all possible units from file.
130 * **missing_ids** - *string* - Determines what happens if a mapping fails or unit_id does not exists. Current options: fail (*default*), warn, ignore.
131 * **save_map** - *string* - A path to a csv file in the output folder where, if specified, will save the units_maps of current simulation. Will overwrite existing file if exists.
132 * **interval** - *list[float (ms), float (ms)]* or *dict* or *list[intervals or dict]* - Filters spike-times in the NWB **input** file, using using a [*start*, *stop*] interval (ms), or if a dictionary is used tries to parse interval from the NWB intervals table. By default will shift interval to time-step 0.0 ms in simulation, use **simulation_offset** to shift to anther value.
133 * **simulation_offset** - *float* - When using the **interval** option to filter spike-times, determines offset for simulation. Default value 0.0.
134
135
136 Advanced Features
137 #################
138
139 Mapping and Filtering NWB units
140 ===============================
141
142 In the simulation each **unit** in NWB must be mapped onto a SONATA **node** for it's spike trains to be used. The `mapping` and `units` options
143 work in conjunction to control how this is done.
144
145 sample and sample_with_replacement
146 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
147
148 To randomly assign unit_ids to node_ids you can use values "sample" or "sample_with_replacement" in the `mapping` option. The former is the same
149 as sample without replacement, which may cause issues if there are fewer units than nodes.
150
151 By default all units in the NWB file will be used, but you can use the `units` option to filter out only those units which meet a certain critera.
152 It must be a dictionary of conditionals that will preform a intersection (eg AND operation). For example to get only units with a "good" value in
153 the "quality" column and comes from channel_ids in 850261194-850261196:
154
155 .. code:: json
156
157 {
158 "mapping": "sample",
159 "units": {
160 "quality": "good",
161 "channel_id": [850261194, 850261195, 850261196]
162 }
163 }
164
165 If you need more advanced comparisions you can specify the column query in the format `"query": {"column": <col>, "operation": <operator>, "value": <value>}`.
166 For example to get only VISp units that having "firing_rate" between 5.0 and 10.0 Hz:
167
168 .. code:: json
169
170 {
171 "mapping": "sample_with_replacement",
172 "units": {
173 "location": "VISp",
174 "fr_gt": {"column": "firing_rate", "operation": ">=", "value": 5.0},
175 "fr_lt": {"column": "firing_rate", "operation": "<=", "value": 10.0}
176 }
177 }
178
179
180 Note: The options available for filter will vary from NWB file to NWB file.
181
182
183 units_map
184 ^^^^^^^^^
185
186 If you know how each unit should be mapped to each node and stored in a csv file, then you can use the "units_map" value in the `mapping` option.
187
188 .. code:: json
189
190 {
191 "mapping": "units_map",
192 "units": "/path/to/units2nodes_map.csv"
193 }
194
195 The *units2nodes_map.csv* is a space-separated csv file where each row contains a SONATA node identifier (Population + node_id) and a corresponding
196 units_id from a NWB file with manditory columns **population**, **node_ids**, **unit_ids**
197
198 .. list-table:: units2nodes_map.csv
199 :widths: 25 25 25
200 :header-rows: 1
201
202 * - population
203 - node_ids
204 - unit_ids
205 * - LGN
206 - 0
207 - 951090537
208 * - LGN
209 - 1
210 - 951090804
211 * - LGN
212 - 2
213 - 951091012
214 * - ...
215 - ...
216 - ...
217
218
219 Filtering Spike Times by Intervals and Epochs
220 =============================================
221
222 In-vio Session data typically are much longer and contains much more stimuli/epochs than that which is of interest to our own simulations. To simulate
223 we can use the `interval` option to only use spikes that occur during a period of interest.
224
225 .. figure:: _static/images/ecephys_probe_intervals.png
226 :scale: 100%
227
228
229 By default the `ecephys_probe` module will shift it to start at zero. So for example if the time-window is set to between 10 and 20 seconds, then a
230 spike that is recorded at 11.5 seconds in the NWB will now occur at time 1.5 seconds in the simulation. You can use the `simulation_offset` option
231 to adjust where spikes starts in the simulation.
232
233 intervals ranges
234 ^^^^^^^^^^^^^^^^
235
236 The easies way to to specify a time-window is to specify a range [*start_time*, *stop_time*] in milliseconds. For example if you know the period of
237 interest you are interested occurs in the probe data between minutes 30 to 32
238
239 .. code:: json
240
241 "interval": [1800000.0, 1920000.0]
242
243 If you have multiple NWB files, each one with a unique period of interest, you can also pass in a list of ranges
244
245
246 .. code:: json
247
248 {
249 "input_file": ["./session_71509.nwb", "./session_73312.nwb", "./session_89923.nwb"],
250 "interval": [[10000, 12000], [15000, 17000], [24000, 26000]]
251 }
252
253
254 Intervals and Epoch Tables (aka Stimulus Table)
255 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
256
257 NWB session files can contain intervals/epochs tables that can tell you the exact time during a session a stimulus or condition was being presented
258 to the subject. If instead of passing an a time range to `interval`, you pass in a `dictionary`, the `ecephys_probe` module will automatically try
259 to create a time-window filter using the columns from the intervals table.
260
261 For example, if you want to get spikes that only occurred when a drifting gratings was presented to subject with a specific a 90 degrees orientation
262 and < 5 Hz frequency:
263
264 .. code:: json
265
266 "interval": {
267 "stimulus_name": "drifting_gratings",
268 "temporal_frequency": {"column": "temporal_frequency", "operation": "<", "value": 5.0},
269 "orientation": 90.0
270 },
271
272 Only the "stimulus_name" value is required. The rest of the options availble to filter on the interval or epoch will vary from both NWB to NWB and
273 from each interval to interval.
274
275 As with interval ranges, when working with multiple NWB files you can pass in a list a interval filters dictionaries.
276
277 One important issue to note is that often a query on the stimulus table will return multiple intervals. For instance drifting gratings with orientation
278 90deg and freq < 5 Hz may be presented to subject multiple times over the full session. By default the module will select the first one, but you
279 can use special option "interval_index" to get a specific one. For example to get the fifth time a drifting grating is presented:
280
281 .. code-block:: json
282
283 "interval": {
284 "interval_name": "drifting_gratings",
285 "interval_index": 5
286 }
287
288 By default the value is set to 0 (the first time that particular interval occurs). But you can also pass in options 'random' (select one interval at
289 random), 'first', 'last', or 'full_range' (returns the minimum start-time and maximum stop-time over all intervals).
290
291
292 Individual unit interval
293 ^^^^^^^^^^^^^^^^^^^^^^^^
294
295 It is also possible to assign each indivdiual unit_id a unique interval of interest when using the "units_maps" option for in the `mapping` attribute.
296 To do so just update the units-to-nodes csv file to include columns **start_times** and **stop_times**, both in units of milliseconds:
297
298 .. list-table:: units2nodes_map.csv
299 :widths: 25 25 25 25 25
300 :header-rows: 1
301
302 * - population
303 - node_ids
304 - unit_ids
305 - start_times
306 - stop_times
307 * - LGN
308 - 0
309 - 951090537
310 - 10034.0
311 - 13124.0
312 * - LGN
313 - 1
314 - 951090804
315 - 11130.5
316 - 12999.8
317 * - LGN
318 - 2
319 - 951091012
320 - 500.0
321 - 2501.3
322 * - ...
323 - ...
324 - ...
325 - ...
326 - ...
327
328
329
330
331 Using the AllenSDK to download Neuropixels data
332 ###############################################
333
334
335