| 1 |
Opening data externally |
| 2 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 3 |
.. _raw_external_data: |
| 4 |
|
| 5 |
This tutorial covers how to open data saved from RAW in other programs. |
| 6 |
This tutorial won't focus on how to use different plotting software, but rather |
| 7 |
on the formatting of data from RAW and how to open it in common programs like |
| 8 |
Excel. |
| 9 |
|
| 10 |
The written version of the tutorial follows. |
| 11 |
|
| 12 |
|
| 13 |
Opening .dat files |
| 14 |
************************* |
| 15 |
|
| 16 |
RAW saves profile data, including the q, I(q), and uncertainty data in .dat files. |
| 17 |
Note that .dat is a common extension for scattering profile data, and other programs, |
| 18 |
such as Primus, also produce .dat files which may have slightly different formats. |
| 19 |
|
| 20 |
File format |
| 21 |
############## |
| 22 |
|
| 23 |
These .dat files are standard text files with space separated values. They |
| 24 |
consist of a header (and possibly a footer), which are marked out by "#" |
| 25 |
starting the header and footer lines. The data is saved in three columns |
| 26 |
with space separation, and the numbers are in scientific format, e.g. 1.23E-04. |
| 27 |
The scattering profile data has a ``# Q I(Q) Error`` |
| 28 |
header line, and three columns which are: |
| 29 |
|
| 30 |
#. **Q** - The experimental q vector |
| 31 |
#. **I(Q)** - The experimental intensities |
| 32 |
#. **Error** - The experimental uncertainty |
| 33 |
|
| 34 |
A large amount of data, including analysis data is saved in either the footer |
| 35 |
(by default) or the header. Regardless of physical location in the file the |
| 36 |
start of this information is distinguished by a "### HEADER:" line. After that, |
| 37 |
if the leading "#" marks are removed from each line the extra data is in json |
| 38 |
format. |
| 39 |
|
| 40 |
The actual header always contains the number of data points and the column headings. |
| 41 |
|
| 42 |
Opening .dat files in Excel |
| 43 |
############################## |
| 44 |
|
| 45 |
RAW .dat files can be opened in Excel (or Libre Office or similar). From there |
| 46 |
it's easy to save the three column data in whatever format you want for further |
| 47 |
plotting or analysis. |
| 48 |
|
| 49 |
#. Open Excel. |
| 50 |
|
| 51 |
#. Create a blank workbook. |
| 52 |
|
| 53 |
#. Choose "File->Import->Text file" (or the data ribbon "Get External Data-> |
| 54 |
From Text" button) and select a RAW .dat file. |
| 55 |
|
| 56 |
#. In the first screen of the import wizard select "Delimited" and click "Next". |
| 57 |
|
| 58 |
|import_dat_excel1_png| |
| 59 |
|
| 60 |
#. In the second screen of the import wizard check "Space". |
| 61 |
|
| 62 |
|import_dat_excel2_png| |
| 63 |
|
| 64 |
#. Click "Finish". |
| 65 |
|
| 66 |
#. In the next window that appears select the location and click "Import". |
| 67 |
|
| 68 |
#. The data will be imported. Note that the column headings will be shifted |
| 69 |
over by one column, the first column is Q, the second I(q) and the third |
| 70 |
Error. |
| 71 |
|
| 72 |
|import_dat_excel3_png| |
| 73 |
|
| 74 |
|
| 75 |
Opening .out files |
| 76 |
******************* |
| 77 |
|
| 78 |
RAW saves IFT data, including the P(r) function and fit to the data, produced |
| 79 |
by GNOM in .out files, which is the output format from the GNOM command line |
| 80 |
program. |
| 81 |
|
| 82 |
File format |
| 83 |
############### |
| 84 |
|
| 85 |
The .out file is a standard text file with space separated values. It contains |
| 86 |
four sections delineated by section headers like: |
| 87 |
``#### Configuration ####``\. |
| 88 |
The numbers for the data in columns are in scientific format, e.g. 1.23E-04. |
| 89 |
The first section is "Configuration", which gives input parameters used to create |
| 90 |
the P(r) function. The second section is "Results", which gives regularization, |
| 91 |
perceptual criteria, real space Rg and I(0) and total estimate results. The |
| 92 |
third section is "Experimental Data and Fit", which consists of 5 columns containing |
| 93 |
the input experimental data and the scattering profile generated as a Fourier |
| 94 |
transform of the P(r) function, which should fit the data (for a good P(r) function). |
| 95 |
The column headers are: |
| 96 |
|
| 97 |
#. **S** - The q vector extrapolated to q=0 |
| 98 |
#. **J Exp** - The experimental intensities |
| 99 |
#. **Error** - The experimental uncertainty |
| 100 |
#. **J Reg** - The scattering profile that is the Fourier transform of the P(r) |
| 101 |
function over the experimental q range (also called the regularized |
| 102 |
intensity) |
| 103 |
#. **I Reg** - The scattering profile that is the Fourier transform of the P(r) |
| 104 |
function over the extrapolated q range (also called the regularized |
| 105 |
intensity) |
| 106 |
|
| 107 |
The fourth section is "Real Space Data", which consists of the P(r) function. |
| 108 |
The columns are: |
| 109 |
|
| 110 |
#. **R** - The real space distance |
| 111 |
#. **P(R)** - The value of the pair distance distribution function at |
| 112 |
a given R value |
| 113 |
#. **ERROR** - The uncertainty in the P(r) value |
| 114 |
|
| 115 |
|
| 116 |
Opening .out files in Excel |
| 117 |
############################### |
| 118 |
|
| 119 |
RAW .out files can be opened in Excel (or Libre Office or similar). From there |
| 120 |
it's easy to save the data in whatever format you want for further |
| 121 |
plotting or analysis. The import process is basically the same as for .dat files, |
| 122 |
above: |
| 123 |
|
| 124 |
#. Open Excel. |
| 125 |
|
| 126 |
#. Create a blank workbook. |
| 127 |
|
| 128 |
#. Choose "File->Import->Text file" (or the data ribbon "Get External Data-> |
| 129 |
From Text" button) and select a GNOM .out file. |
| 130 |
|
| 131 |
#. In the first screen of the import wizard select "Delimited" and click "Next". |
| 132 |
|
| 133 |
#. In the second screen of the import wizard check "Space". |
| 134 |
|
| 135 |
#. Click "Finish". |
| 136 |
|
| 137 |
#. In the next window that appears select the location and click "Import". |
| 138 |
|
| 139 |
#. In the imported data, some of the I Reg values will be in the wrong column, |
| 140 |
due to how separators are handled. Scroll down to the experimental data section. |
| 141 |
Select the Data in the second column from the top to just above where the |
| 142 |
next three columns start. |
| 143 |
|
| 144 |
|import_out_excel1_png| |
| 145 |
|
| 146 |
#. Cut that data and paste it at the top of the fifth column. |
| 147 |
|
| 148 |
|import_out_excel2_png| |
| 149 |
|
| 150 |
#. The header labels are not in the correct columns, but the five data columns |
| 151 |
in the experimental section now correspond to those given above: S, J Exp, |
| 152 |
Error, J Reg, I Reg respectively. |
| 153 |
|
| 154 |
#. Scroll down further to find the three column P(r) data with the correct |
| 155 |
column headers. |
| 156 |
|
| 157 |
|import_out_excel3_png| |
| 158 |
|
| 159 |
|
| 160 |
Opening .ift files |
| 161 |
******************* |
| 162 |
|
| 163 |
RAW saves IFT data produced by BIFT in .ift files. |
| 164 |
|
| 165 |
File format |
| 166 |
############## |
| 167 |
|
| 168 |
These .ift files are standard text files with space separated values. |
| 169 |
The numbers for the data in columns are in scientific format, e.g. 1.23E-04. |
| 170 |
These files consist of several different sections marked by columns headers on |
| 171 |
lines starting with "#". The whole file overall starts with a "# BIFT" line to |
| 172 |
identify the type of IFT. The first section is the P(R) function and is started |
| 173 |
with the ``# R P(R) Error`` header. The columns are: |
| 174 |
|
| 175 |
|
| 176 |
#. **R** - The real space distance |
| 177 |
#. **P(R)** - The value of the pair distance distribution function at |
| 178 |
a given R value |
| 179 |
#. **Error** - The uncertainty in the P(r) value |
| 180 |
|
| 181 |
The second section is the experimental data and the scattering profile generated |
| 182 |
as a Fourier transform of the P(r) function, which should fit the data (for a |
| 183 |
good P(r) function). It is started with the |
| 184 |
``# Q I(Q) Error Fit`` header. The columns are: |
| 185 |
|
| 186 |
#. **Q** - The experimental q vector |
| 187 |
#. **I(Q)** - The experimental intensities |
| 188 |
#. **Error** - The experimental uncertainty |
| 189 |
#. **Fit** - The scattering profile that is the Fourier transform of the P(r) |
| 190 |
function over the experimental q range (also called the regularized |
| 191 |
intensity) |
| 192 |
|
| 193 |
The third section is the fit/regularized intensity extrapolated to q=0 and |
| 194 |
starts with the |
| 195 |
``# Q_extrap Fit_extrap`` |
| 196 |
header. The columns are: |
| 197 |
|
| 198 |
#. **Q_extrap** - The extrapolated q vector |
| 199 |
#. **Fit_extrap** - The scattering profile that is the Fourier transform |
| 200 |
of the P(r) function over the extrapolated q range (also called the regularized |
| 201 |
intensity) |
| 202 |
|
| 203 |
The fourth section is data about the parameters used and derived values, |
| 204 |
such as the I(0) and Rg, which are saved at the end of the file. This |
| 205 |
information is distinguished by a ``### HEADER:`` line. After that, |
| 206 |
if the leading "#" marks are removed from each line the extra data is in json |
| 207 |
format. |
| 208 |
|
| 209 |
Opening .ift files in Excel |
| 210 |
############################### |
| 211 |
|
| 212 |
RAW .ift files can be opened in Excel (or Libre Office or similar). From there |
| 213 |
it's easy to save the data in whatever format you want for further |
| 214 |
plotting or analysis. The import process is basically the same as for .dat files, |
| 215 |
above: |
| 216 |
|
| 217 |
#. Open Excel. |
| 218 |
|
| 219 |
#. Create a blank workbook. |
| 220 |
|
| 221 |
#. Choose "File->Import->Text file" (or the data ribbon "Get External Data-> |
| 222 |
From Text" button) and select an .ift file. |
| 223 |
|
| 224 |
#. In the first screen of the import wizard select "Delimited" and click "Next". |
| 225 |
|
| 226 |
#. In the second screen of the import wizard check "Space". |
| 227 |
|
| 228 |
#. Click "Finish". |
| 229 |
|
| 230 |
#. In the next window that appears select the location and click "Import". |
| 231 |
|
| 232 |
#. The data will be imported. Note that the column headings will be shifted |
| 233 |
over by one column. For example, for the P(r) data, the first column is |
| 234 |
the R data, the second the P(R) data, and the third the Error data. |
| 235 |
|
| 236 |
#. Scroll down through the spreadsheet to see all of the imported data. |
| 237 |
|
| 238 |
|
| 239 |
Opening .csv files |
| 240 |
******************* |
| 241 |
|
| 242 |
RAW saves most exported data and analysis in .csv (comma separated value) format. |
| 243 |
|
| 244 |
File format |
| 245 |
############## |
| 246 |
|
| 247 |
These .csv files are standard text files with comma separated values. Files |
| 248 |
generated from RAW will have an header information such as column headings |
| 249 |
marked with a "#" at the start of the line. |
| 250 |
|
| 251 |
Opening .csv files in Excel |
| 252 |
############################# |
| 253 |
|
| 254 |
.csv files generally be opened in Excel (or Libre Office or similar) by double |
| 255 |
clicking on the file. From there it's easy to save the data in whatever format |
| 256 |
you want for further plotting or analysis. If double clicking doesn't work, |
| 257 |
the import process is basically the same as for .dat files, above: |
| 258 |
|
| 259 |
#. Open Excel. |
| 260 |
|
| 261 |
#. Create a blank workbook. |
| 262 |
|
| 263 |
#. Choose "File->Import->Text file" (or the data ribbon "Get External Data-> |
| 264 |
From Text" button) and select an .ift file. |
| 265 |
|
| 266 |
#. In the first screen of the import wizard select "Delimited" and click "Next". |
| 267 |
|
| 268 |
#. In the second screen of the import wizard check "Comma". |
| 269 |
|
| 270 |
#. Click "Finish". |
| 271 |
|
| 272 |
#. In the next window that appears select the location and click "Import". |
| 273 |
|
| 274 |
|
| 275 |
.. |import_dat_excel1_png| image:: images/import_dat_excel1.png |
| 276 |
:target: ../_images/import_dat_excel1.png |
| 277 |
:width: 500 px |
| 278 |
|
| 279 |
.. |import_dat_excel2_png| image:: images/import_dat_excel2.png |
| 280 |
:target: ../_images/import_dat_excel2.png |
| 281 |
:width: 500 px |
| 282 |
|
| 283 |
.. |import_dat_excel3_png| image:: images/import_dat_excel3.png |
| 284 |
:target: ../_images/import_dat_excel3.png |
| 285 |
:width: 500 px |
| 286 |
|
| 287 |
.. |import_out_excel1_png| image:: images/import_out_excel1.png |
| 288 |
:target: ../_images/import_out_excel1.png |
| 289 |
|
| 290 |
.. |import_out_excel2_png| image:: images/import_out_excel2.png |
| 291 |
:target: ../_images/import_out_excel2.png |
| 292 |
|
| 293 |
.. |import_out_excel3_png| image:: images/import_out_excel3.png |
| 294 |
:target: ../_images/import_out_excel3.png |