Debian Codegraph @

docs/autodocs/source/simulators.rst

raw
1 Simulation Engines
2 ==================
3 .. toctree::
4 :maxdepth: 2
5 :titlesonly:
6 :hidden:
7
8 bionet
9 pointnet
10 popnet
11 filternet
12
13
14 .. figure:: _static/images/bmtk_architecture_simulator_highlight.jpg
15 :scale: 40%
16
17 The :py:mod:`bmtk.simulator` module contains classes for simulating a network under a variety of different simulators.
18 BMTK doesn't actually do the simulation itself, but processes the inputs and passes it off to an existing engine. At the
19 moment there are four different simulation engines for running a network at different levels-of-resolution:
20
21 * `BioNet <./bionet>`_ - Uses `NEURON <https://neuron.yale.edu>`_ to run biophysically realistic models
22 * `PointNet <./pointnet>`_ - Uses `NEST <https://www.nest-simulator.org/>`_ to run point-neuron based models
23 * `PopNet <./popnet>`_ - Uses `DiPDE <https://github.com/AllenInstitute/dipde>`_ to run population-based firing rate models.
24 * `FilterNet <./filternet>`_ - Uses Filter based units, that uses a liner-nonlinear-poisson cascading models, base on the
25 mammalian LGN, to convert inputs into spikes.
26
27 .. figure:: _static/images/brunel_comparisons.jpg
28 :scale: 40%
29
30 usually capture a neuron's membrane potential trace (when the model_template supports it) that doesn’t isn't possible on
31 a population or filter-based level of resolution. And specific dynamics may occur at one level and not another. Thus it
32 is important for the modeler to choose what levels they use. However running simulations across the different engines is
33 done in the same way. The following sections provide a high level overview of how to set up and run network simulations
34 across all the engines. Functionality that is unique to a specific engine is covered more in-depth in the documentation
35 for that engine.
36
37 In most cases running a simulation doesn’t require any programming - and at most just the ability to execute a
38 command-line script. The process for setting up and running a simulation usually involves three steps
39
40 1. Setting up the environment, a folder with the necessary files and scripts.
41
42 2. Modifying the configuration files with a text-editor
43
44 3. Executing the simulation
45
46 The Simulation Environment
47 --------------------------
48 The standard BMTK directory structure for running one or more simulations will typically have all the necessary files
49 contained within their own directory as such:
50
51 .. figure:: _static/images/bmtk_sim_env.png
52 :scale: 40%
53
54 * **network/** - directory containing the SONATA Network files, either built using the `NetworkBuilder <./builder.html>`_
55 or provided to us from another source.
56
57 * **components/** - Will usually contain neuron and synaptic parameter files, morphologies, NEURON hoc mechanisms.
58 In general files, external files used to instantiate the nodes and edges of the network.
59
60 * **inputs/** - Often contain spike-trains, movies, images and other stimuli to drive our network. Not always required.
61
62 * **circuit_config.json** - configuration file containing information for instantiating the network.
63
64 * **simulation_config.json** - configuration file with information on how to run the simulation, including the input
65 and output of the simulation.
66
67 * **run_simulation.py** - a python script that will actually run the simulation.
68
69 The names of the files and folders can be changed as the modeler desires. Nor is it required or always beneficial that
70 everything be contained within the same directory. For example different networks may share the same neuron/synaptic
71 component files, thus it would make sense to move the *components* directory to a location that can be accessed by
72 all simulations. As described below the location of the directories and files are specified in the configuration files.
73
74 Creating the environment
75 ++++++++++++++++++++++++
76 A good way to create a working simulation directory is to copy an existing one and make modifications as needed. The
77 `BMTK <https://github.com/AllenInstitute/bmtk>`_ and `SONATA <https://github.com/AllenInstitute/sonata>`_ github page
78 contains multiple examples.
79
80 BMTK also contains a script :py:func:`bmtk.utils.sim_setup` that will automatically create a base-line simulation
81 environment for you. To create the directory structure for a BioNet simulation called my_sim, including the necessary
82 config and run scripts, run the following in the command-line:
83
84 .. code:: bash
85
86 $ python -m bmtk.utils.sim_setup bionet my_sim
87
88 You may also replace **bionet** with either **pointnet**, **popnet**, or **filternet**. Once it *my_sim/* directory has
89 been created the modeler will need to move the necessary network, components, and input files to their correct place. Then
90 modify the configuration, using a text editor, as required. And finally the network can be run. See below.
91
92 The sim_setup script also contains many extra options, see the help menu for full list:
93
94 .. code:: bash
95
96 $ python -m bmtk.utils.sim_setup -h
97
98
99 Configuration Files
100 -------------------
101 Under most circumstances setting up and/or modifying a simulation involves adding and altering values within the JSON
102 configuration files. The SONATA standard specifies two configuration files; a circuit_config.json containing information
103 about the network and component files. And a simulation_config.json used for actually running the full circuit. BMTK
104 allows modelers to combine the two configs into one (but other SONATA support software may not).
105
106 The following breaks down the different sections found in the SONATA configs. See the
107 `SONATA documentation <https://github.com/AllenInstitute/sonata/blob/master/docs/SONATA_DEVELOPER_GUIDE.md>`_ for
108 more details. Some simulation engines may allow for additional options that will be described in their respective docs.
109
110 manifest
111 ++++++++
112 The “manifest” section is optional and allows modelers to define aliases within the config, which are signified by the
113 dollar-sign like shell variables; eg ``$VARNAME`` or ``${VARNAME}``. For example if the manifest contains the following:
114
115 .. code:: json
116
117 {
118 "manifest": {
119 "$COMP_DIR": "/home/ahodgkin/workspace/shared/bionet/components":
120 },
121
122 "components": {
123 "morphologies_dir": "$COMP_DIR/morphologies",
124 "synaptic_models_dir": "$COMP_DIR/synaptic_models"
125 }
126 }
127
128 networks
129 ++++++++
130 contains locations of the node and edges files used to build the circuit, such that multiple node and edge files may be
131 combined into one circuit. For example the following contains V1 network, with their recurrent connections
132 (v1_v1_edges\*) plus input from LGN nodes (lgn_v1_edges\*)
133
134 .. code:: json
135
136 {
137 "networks": {
138 "nodes": [
139 {
140 "nodes_file": "$NETWORK_DIR/v1_nodes.h5",
141 "node_types_file": "$NETWORK_DIR/v1_node_types.csv"
142 },
143 {
144 "nodes_file": "$NETWORK_DIR/lgn_nodes.h5",
145 "node_types_file": "$NETWORK_DIR/lgn_node_types.csv"
146 }
147 ],
148 "edges": [
149 {
150 "edges_file": "$NETWORK_DIR/v1_v1_edges.h5",
151 "edge_types_file": "$NETWORK_DIR/v1_v1_edge_types.csv"
152 },
153 {
154 "edges_file": "$NETWORK_DIR/lgn_v1_edges.h5",
155 "edge_types_file": "$NETWORK_DIR/lgn_v1_edge_types.csv"
156 }
157 ]
158 }
159 }
160
161 Changing the network can be done by removing or adding nodes or edges. BMTK also has the “enabled” option (not a part
162 of SONATA)
163
164 .. code:: json
165
166 {
167 "edges_file": "$NETWORK_DIR/lgn_v1_edges.h5",
168 "edge_types_file": "$NETWORK_DIR/lgn_v1_edge_types.csv",
169 "enabled": false
170 }
171
172 components
173 ++++++++++
174 links to external directories and files required to fully instantiate all the models and parameters of the network,
175 see `SONATA circuit config documentation <https://github.com/AllenInstitute/sonata/blob/master/docs/SONATA_DEVELOPER_GUIDE.md#tying-it-all-together---the-networkcircuit-config-file>`_
176 for a full description
177
178 .. code:: json
179
180 {
181 "components": {
182 "morphologies_dir": "./components/morphologies",
183 "synaptic_models_dir": "./components/synaptic_models",
184 "mechanisms_dir":"./components/mechanisms",
185 "point_neuron_models_dir": "./components/point_neuron_models_dir"
186 }
187 }
188
189 run
190 +++
191 contains simulation parameters. most importantly are
192
193 * tstart: simulation start time in ms (default 0.0)
194 * stop: final simulation time in ms
195 * dt: time step parameter
196 * overwrite_output_dir: overwrites any previous ran results
197
198 .. code:: json
199
200 {
201 "run": {
202 "tstop": 3000.0,
203 "dt": 0.1,
204 "overwrite_output_dir": true
205 }
206 }
207
208
209 output
210 ++++++
211 Sections contain general information about where to output the results on the simulation. When the simulation starts
212 the output_dir will be automatically created (and will overwrite existing output if overwrite_output_dir is true) as
213 well as the log_file. By default BMTK will always record and save the spikes of all the nodes of the simulation
214 (see "reports" section for how to change this) into the "spikes_file".
215
216 .. code:: json
217
218 {
219 "output":{
220 "output_dir": "./output",
221 "log_file": "log.txt",
222 "spikes_file": "spikes.h5",
223 "spikes_sort_order": "time"
224 }
225 }
226
227 inputs
228 ++++++
229 Contains different inputs that will drive the simulation. In BioNet and PointNet usually the inputs involves either
230 spike trains injected into the network using external **virtual** nodes, or the use of current clamps. The following
231 example does both:
232
233 .. code:: json
234
235 {
236 "inputs": {
237 "virtual_spikes": {
238 "input_type": "spikes",
239 "module": "h5",
240 "node_set": {"population": "LGN"},
241 "input_file": "$INPUT_DIR/lgn_spikes.h5"
242 },
243 "current_clamp": {
244 "input_type": "current_clamp",
245 "module": "IClamp",
246 "node_set": {"node_ids": [10, 20, 30, 100]},
247 "amp": 0.1500,
248 "delay": 500.0,
249 "duration": 500.0
250 }
251 }
252 }
253
254 The ``input_type`` and ``module`` parameters for each input describes the method and type of input. The ``node_set``,
255 described below, filters the cells for which the stimulus will be applied too. Other types of stimuli are specific to
256 only certain engines; like extracellular potentiation in BioNet or movies in FilterNet.
257
258
259 reports
260 +++++++
261 Contains information about what variables will be recorded during the simulation. For example to record membrane
262 potential in all the biophysically detailed cells in the V1 population:
263
264 .. code:: json
265
266 {
267 "reports": {
268 "membrane_potential": {
269 "cells": {"population": "V1", "model_type": "biophysical"},
270 "variable_name": "v",
271 "module": "membrane_report",
272 "file_name": "cell_vars.h5",
273 "sections": "soma"
274 }
275 }
276 }
277
278 By default BMTK will record the spike-trains of all non-virtual cells as specified in the ``outputs`` section, so it is not
279 necessary to create a ``"module": "spike_trians"`` report. But you may if, for example, you only want to record for
280 part of the full simulation time or record from a subset of all cells.
281
282
283 node_sets
284 +++++++++
285 `node_sets <https://github.com/AllenInstitute/sonata/blob/master/docs/SONATA_DEVELOPER_GUIDE.md#node-sets-file>`_
286 are filters for specific nodes within the circuit, mainly used to determine which cells to apply an "input" or a
287 "report". For example say we want to split the spike-train output of V1’s pvalb and pyramidal cells into different
288 reports
289
290 .. code:: json
291
292 {
293 "node_sets": {
294 "pvalb_cells": {
295 "population": "V1",
296 "cell_type": "pvalb"
297 },
298 "pyr_cells":: {
299 "population": "V1",
300 "cell_type": "pyramidal"
301 },
302 },
303
304 "reports": {
305 "pvalb_spikes": {
306 "cells": "pvalb_cells",
307 "module": "spikes_report",
308 "file_name": "pvalb_spikes.h5",
309 "sort_by": "node_id"
310 },
311 "pyr_spikes": {
312 "cells": "pyr_cells",
313 "module": "spikes_report",
314 "file_name": "pyr_spikes.h5",
315 "sort_by": "node_id"
316 }
317 }
318 }
319
320 ``node_sets`` may also be a reference to an external json file.
321
322 If the node_set value is just a name, ie ``"cells": "V1"`` then it will automatically include cells in the “V1”
323 population. You may also reference individual nodes by their “node_id” - ``"cells": [0, 1, 2]``
324
325
326 Running a Simulation
327 --------------------
328 To run a simulation, open a command line and execute the run_<simulation>.py script:
329
330 .. code:: bash
331
332 $ python run_simulation.py simulation_config.json
333
334 Output will be written to the “output_dir” as specified in the configuration, including a running log file.
335
336 For PointNet, PopNet, and FilterNet the simulations can be run using parallel cores using MPI with the following command:
337
338 .. code:: bash
339
340 $ mpirun -np <N> python run_simulation.py simulation_config.json
341
342 For BioNet NEURON requires a slightly different way:
343
344 .. code:: bash
345
346 $ mpirun -np <N> nrniv -mpi -python run_simulation.py simulation_config.json
347