Debian Codegraph @

docs/source/tutorial/s4_external_data.rst

raw
1 Opening data externally
2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3 .. _raw_external_data:
4
5 This tutorial covers how to open data saved from RAW in other programs.
6 This tutorial won't focus on how to use different plotting software, but rather
7 on the formatting of data from RAW and how to open it in common programs like
8 Excel.
9
10 The written version of the tutorial follows.
11
12
13 Opening .dat files
14 *************************
15
16 RAW saves profile data, including the q, I(q), and uncertainty data in .dat files.
17 Note that .dat is a common extension for scattering profile data, and other programs,
18 such as Primus, also produce .dat files which may have slightly different formats.
19
20 File format
21 ##############
22
23 These .dat files are standard text files with space separated values. They
24 consist of a header (and possibly a footer), which are marked out by "#"
25 starting the header and footer lines. The data is saved in three columns
26 with space separation, and the numbers are in scientific format, e.g. 1.23E-04.
27 The scattering profile data has a ``# Q I(Q) Error``
28 header line, and three columns which are:
29
30 #. **Q** - The experimental q vector
31 #. **I(Q)** - The experimental intensities
32 #. **Error** - The experimental uncertainty
33
34 A large amount of data, including analysis data is saved in either the footer
35 (by default) or the header. Regardless of physical location in the file the
36 start of this information is distinguished by a "### HEADER:" line. After that,
37 if the leading "#" marks are removed from each line the extra data is in json
38 format.
39
40 The actual header always contains the number of data points and the column headings.
41
42 Opening .dat files in Excel
43 ##############################
44
45 RAW .dat files can be opened in Excel (or Libre Office or similar). From there
46 it's easy to save the three column data in whatever format you want for further
47 plotting or analysis.
48
49 #. Open Excel.
50
51 #. Create a blank workbook.
52
53 #. Choose "File->Import->Text file" (or the data ribbon "Get External Data->
54 From Text" button) and select a RAW .dat file.
55
56 #. In the first screen of the import wizard select "Delimited" and click "Next".
57
58 |import_dat_excel1_png|
59
60 #. In the second screen of the import wizard check "Space".
61
62 |import_dat_excel2_png|
63
64 #. Click "Finish".
65
66 #. In the next window that appears select the location and click "Import".
67
68 #. The data will be imported. Note that the column headings will be shifted
69 over by one column, the first column is Q, the second I(q) and the third
70 Error.
71
72 |import_dat_excel3_png|
73
74
75 Opening .out files
76 *******************
77
78 RAW saves IFT data, including the P(r) function and fit to the data, produced
79 by GNOM in .out files, which is the output format from the GNOM command line
80 program.
81
82 File format
83 ###############
84
85 The .out file is a standard text file with space separated values. It contains
86 four sections delineated by section headers like:
87 ``#### Configuration ####``\.
88 The numbers for the data in columns are in scientific format, e.g. 1.23E-04.
89 The first section is "Configuration", which gives input parameters used to create
90 the P(r) function. The second section is "Results", which gives regularization,
91 perceptual criteria, real space Rg and I(0) and total estimate results. The
92 third section is "Experimental Data and Fit", which consists of 5 columns containing
93 the input experimental data and the scattering profile generated as a Fourier
94 transform of the P(r) function, which should fit the data (for a good P(r) function).
95 The column headers are:
96
97 #. **S** - The q vector extrapolated to q=0
98 #. **J Exp** - The experimental intensities
99 #. **Error** - The experimental uncertainty
100 #. **J Reg** - The scattering profile that is the Fourier transform of the P(r)
101 function over the experimental q range (also called the regularized
102 intensity)
103 #. **I Reg** - The scattering profile that is the Fourier transform of the P(r)
104 function over the extrapolated q range (also called the regularized
105 intensity)
106
107 The fourth section is "Real Space Data", which consists of the P(r) function.
108 The columns are:
109
110 #. **R** - The real space distance
111 #. **P(R)** - The value of the pair distance distribution function at
112 a given R value
113 #. **ERROR** - The uncertainty in the P(r) value
114
115
116 Opening .out files in Excel
117 ###############################
118
119 RAW .out files can be opened in Excel (or Libre Office or similar). From there
120 it's easy to save the data in whatever format you want for further
121 plotting or analysis. The import process is basically the same as for .dat files,
122 above:
123
124 #. Open Excel.
125
126 #. Create a blank workbook.
127
128 #. Choose "File->Import->Text file" (or the data ribbon "Get External Data->
129 From Text" button) and select a GNOM .out file.
130
131 #. In the first screen of the import wizard select "Delimited" and click "Next".
132
133 #. In the second screen of the import wizard check "Space".
134
135 #. Click "Finish".
136
137 #. In the next window that appears select the location and click "Import".
138
139 #. In the imported data, some of the I Reg values will be in the wrong column,
140 due to how separators are handled. Scroll down to the experimental data section.
141 Select the Data in the second column from the top to just above where the
142 next three columns start.
143
144 |import_out_excel1_png|
145
146 #. Cut that data and paste it at the top of the fifth column.
147
148 |import_out_excel2_png|
149
150 #. The header labels are not in the correct columns, but the five data columns
151 in the experimental section now correspond to those given above: S, J Exp,
152 Error, J Reg, I Reg respectively.
153
154 #. Scroll down further to find the three column P(r) data with the correct
155 column headers.
156
157 |import_out_excel3_png|
158
159
160 Opening .ift files
161 *******************
162
163 RAW saves IFT data produced by BIFT in .ift files.
164
165 File format
166 ##############
167
168 These .ift files are standard text files with space separated values.
169 The numbers for the data in columns are in scientific format, e.g. 1.23E-04.
170 These files consist of several different sections marked by columns headers on
171 lines starting with "#". The whole file overall starts with a "# BIFT" line to
172 identify the type of IFT. The first section is the P(R) function and is started
173 with the ``# R P(R) Error`` header. The columns are:
174
175
176 #. **R** - The real space distance
177 #. **P(R)** - The value of the pair distance distribution function at
178 a given R value
179 #. **Error** - The uncertainty in the P(r) value
180
181 The second section is the experimental data and the scattering profile generated
182 as a Fourier transform of the P(r) function, which should fit the data (for a
183 good P(r) function). It is started with the
184 ``# Q I(Q) Error Fit`` header. The columns are:
185
186 #. **Q** - The experimental q vector
187 #. **I(Q)** - The experimental intensities
188 #. **Error** - The experimental uncertainty
189 #. **Fit** - The scattering profile that is the Fourier transform of the P(r)
190 function over the experimental q range (also called the regularized
191 intensity)
192
193 The third section is the fit/regularized intensity extrapolated to q=0 and
194 starts with the
195 ``# Q_extrap Fit_extrap``
196 header. The columns are:
197
198 #. **Q_extrap** - The extrapolated q vector
199 #. **Fit_extrap** - The scattering profile that is the Fourier transform
200 of the P(r) function over the extrapolated q range (also called the regularized
201 intensity)
202
203 The fourth section is data about the parameters used and derived values,
204 such as the I(0) and Rg, which are saved at the end of the file. This
205 information is distinguished by a ``### HEADER:`` line. After that,
206 if the leading "#" marks are removed from each line the extra data is in json
207 format.
208
209 Opening .ift files in Excel
210 ###############################
211
212 RAW .ift files can be opened in Excel (or Libre Office or similar). From there
213 it's easy to save the data in whatever format you want for further
214 plotting or analysis. The import process is basically the same as for .dat files,
215 above:
216
217 #. Open Excel.
218
219 #. Create a blank workbook.
220
221 #. Choose "File->Import->Text file" (or the data ribbon "Get External Data->
222 From Text" button) and select an .ift file.
223
224 #. In the first screen of the import wizard select "Delimited" and click "Next".
225
226 #. In the second screen of the import wizard check "Space".
227
228 #. Click "Finish".
229
230 #. In the next window that appears select the location and click "Import".
231
232 #. The data will be imported. Note that the column headings will be shifted
233 over by one column. For example, for the P(r) data, the first column is
234 the R data, the second the P(R) data, and the third the Error data.
235
236 #. Scroll down through the spreadsheet to see all of the imported data.
237
238
239 Opening .csv files
240 *******************
241
242 RAW saves most exported data and analysis in .csv (comma separated value) format.
243
244 File format
245 ##############
246
247 These .csv files are standard text files with comma separated values. Files
248 generated from RAW will have an header information such as column headings
249 marked with a "#" at the start of the line.
250
251 Opening .csv files in Excel
252 #############################
253
254 .csv files generally be opened in Excel (or Libre Office or similar) by double
255 clicking on the file. From there it's easy to save the data in whatever format
256 you want for further plotting or analysis. If double clicking doesn't work,
257 the import process is basically the same as for .dat files, above:
258
259 #. Open Excel.
260
261 #. Create a blank workbook.
262
263 #. Choose "File->Import->Text file" (or the data ribbon "Get External Data->
264 From Text" button) and select an .ift file.
265
266 #. In the first screen of the import wizard select "Delimited" and click "Next".
267
268 #. In the second screen of the import wizard check "Comma".
269
270 #. Click "Finish".
271
272 #. In the next window that appears select the location and click "Import".
273
274
275 .. |import_dat_excel1_png| image:: images/import_dat_excel1.png
276 :target: ../_images/import_dat_excel1.png
277 :width: 500 px
278
279 .. |import_dat_excel2_png| image:: images/import_dat_excel2.png
280 :target: ../_images/import_dat_excel2.png
281 :width: 500 px
282
283 .. |import_dat_excel3_png| image:: images/import_dat_excel3.png
284 :target: ../_images/import_dat_excel3.png
285 :width: 500 px
286
287 .. |import_out_excel1_png| image:: images/import_out_excel1.png
288 :target: ../_images/import_out_excel1.png
289
290 .. |import_out_excel2_png| image:: images/import_out_excel2.png
291 :target: ../_images/import_out_excel2.png
292
293 .. |import_out_excel3_png| image:: images/import_out_excel3.png
294 :target: ../_images/import_out_excel3.png