| 1 |
+++ |
| 2 |
title = "Data import" |
| 3 |
weight = 10 |
| 4 |
+++ |
| 5 |
|
| 6 |
## Overview |
| 7 |
|
| 8 |
BornAgain provides functions to read 1D and 2D data fields: |
| 9 |
|
| 10 |
```python |
| 11 |
ba.readData1D(fname) |
| 12 |
ba.readData1D(fname, ftype) |
| 13 |
ba.readData1D(fname, ftype, import_settings) |
| 14 |
|
| 15 |
ba.readData2D(fname) |
| 16 |
ba.readData2D(fname, ftype) |
| 17 |
ba.readData2D(fname, ftype, import_settings) |
| 18 |
``` |
| 19 |
|
| 20 |
The single-argument forms determine the file format from the file name |
| 21 |
extension (see tables below). |
| 22 |
If the name ends with `.gz` or `.bz2`, the file is decompressed on the fly. |
| 23 |
|
| 24 |
The multi-argument forms accept an explicit format constant (`ftype`), |
| 25 |
needed when the file extension does not match the format or is not specific enough. |
| 26 |
A third, optional argument is an `ImportSettings` object |
| 27 |
that specifies in particular the column layout. |
| 28 |
It is always required for 1D CSV files. |
| 29 |
|
| 30 |
## Supported formats |
| 31 |
|
| 32 |
##### 1D formats (for specular reflectometry) |
| 33 |
|
| 34 |
| Format | Extension | Enum constant | |
| 35 |
|--------|-----------|---------------| |
| 36 |
| BornAgain internal text format | `.int` | `ba.bornagain1D` | |
| 37 |
| Motofit format | `.mft` | `ba.mft` | |
| 38 |
| CSV/ASCII column data | any other | `ba.csv1D` | |
| 39 |
|
| 40 |
##### 2D formats (for GISAS/diffraction) |
| 41 |
|
| 42 |
| Format | Extension | Enum constant | |
| 43 |
|--------|-----------|---------------| |
| 44 |
| BornAgain internal text format | `.int` | `ba.bornagain2D` | |
| 45 |
| [TIFF image](tiff) | `.tif`, `.tiff` | `ba.tiff` | |
| 46 |
| BERsans format | `.001` | `ba.nicos2D` | |
| 47 |
| CSV/ASCII matrix data | any other | `ba.csv2D` | |
| 48 |
|
| 49 |
## Reading 1D data |
| 50 |
|
| 51 |
For CSV/ASCII column files, `ImportSettings1D` is always required |
| 52 |
(the single-argument form throws for CSV, as it cannot infer column layout). |
| 53 |
Pass format and settings explicitly: |
| 54 |
|
| 55 |
```python |
| 56 |
flags = ba.ImportSettings1D("2alpha (deg)", "#", "", 1, 2) |
| 57 |
data = ba.readData1D(fname, ba.csv1D, flags) |
| 58 |
``` |
| 59 |
|
| 60 |
For self-describing formats (`.int`, `.mft`), no settings are needed: |
| 61 |
|
| 62 |
```python |
| 63 |
data = ba.readData1D("reflectivity.mft") |
| 64 |
``` |
| 65 |
|
| 66 |
**Examples:** |
| 67 |
[Specular1Par.py][spec1par], |
| 68 |
[Honeycomb_fit.py][honeycomb], |
| 69 |
[Pt_layer_fit.py][ptlayer], |
| 70 |
[TREFF_Ni_film.py][treff]. |
| 71 |
|
| 72 |
##### ImportSettings1D |
| 73 |
|
| 74 |
`ImportSettings1D` describes the column layout of a CSV reflectivity file. |
| 75 |
Column indices count from 1. |
| 76 |
|
| 77 |
```python |
| 78 |
flags = ba.ImportSettings1D( |
| 79 |
xCoord, # coordinate type, e.g. "q_z (1/nm)" or "2alpha (deg)" |
| 80 |
headerPrefix, # prefix that marks header lines, e.g. "#" |
| 81 |
linesToSkip, # line-number pattern to skip, e.g. "1,10-12" |
| 82 |
col_Q, # column number of Q (or angle) |
| 83 |
col_R, # column number of R (intensity) |
| 84 |
col_sR=0, # column of sigma_R (uncertainty), 0 = absent |
| 85 |
col_dQ=0, # column of delta_Q (resolution), 0 = absent |
| 86 |
col_lambda=0, # column of wavelength, 0 = absent |
| 87 |
sort=True, # sort rows by Q |
| 88 |
rm_negative=True, # discard rows with negative Q |
| 89 |
rm_duplications=True # discard rows with duplicate Q |
| 90 |
) |
| 91 |
``` |
| 92 |
|
| 93 |
## Reading 2D data |
| 94 |
|
| 95 |
The simplest call uses automatic format detection from the file name: |
| 96 |
|
| 97 |
```python |
| 98 |
data = ba.readData2D("detector_image.tif.gz") |
| 99 |
``` |
| 100 |
|
| 101 |
An explicit file type can be given as second argument. |
| 102 |
For CSV format, `ImportSettings2D` is also required: |
| 103 |
|
| 104 |
```python |
| 105 |
data = ba.readData2D(fname, ba.csv2D, import_settings) |
| 106 |
``` |
| 107 |
|
| 108 |
**Examples:** |
| 109 |
[expfit_galaxi.py][galaxi] loads a TIFF file and fits a GISAS pattern |
| 110 |
from the GALAXI instrument. |
| 111 |
[fit_gisas.py][fitgisas] loads a BornAgain internal-format file. |
| 112 |
|
| 113 |
##### ImportSettings2D |
| 114 |
|
| 115 |
When reading a 2D CSV file, `ImportSettings2D` controls how axes are parsed: |
| 116 |
|
| 117 |
```python |
| 118 |
settings = ba.ImportSettings2D( |
| 119 |
xCoord, # coordinate type of the x axis, e.g. "phi_f (deg)" |
| 120 |
yCoord, # coordinate type of the y axis, e.g. "alpha_f (deg)" |
| 121 |
has_axes=False, # True if the file contains explicit axis values |
| 122 |
swap_axes=False, # True to swap x and y |
| 123 |
first_row=True, # True if axis values are in the first row (else last) |
| 124 |
first_col=True, # True if axis values are in the first column (else last) |
| 125 |
) |
| 126 |
``` |
| 127 |
|
| 128 |
## Reading using fabio and NumPy |
| 129 |
|
| 130 |
If your instrument produces a format not listed above, read the data |
| 131 |
into a numpy array with a third-party library and wrap it in a |
| 132 |
`Datafield` before passing it to `FitObjective`: |
| 133 |
|
| 134 |
```python |
| 135 |
import fabio # example: EDF/CBF/HDF5 and many more |
| 136 |
import bornagain as ba |
| 137 |
|
| 138 |
img = fabio.open("experimental_data.edf") |
| 139 |
arr = img.data.astype("float64") |
| 140 |
nrows, ncols = arr.shape |
| 141 |
frame = ba.Frame(ba.EquiDivision("x", ncols, 0.0, float(ncols)), |
| 142 |
ba.EquiDivision("y", nrows, 0.0, float(nrows))) |
| 143 |
data = ba.Datafield(frame, list(arr.flatten())) |
| 144 |
``` |
| 145 |
|
| 146 |
Then pass the `Datafield` to the fit engine: |
| 147 |
|
| 148 |
```python |
| 149 |
fit_objective = ba.FitObjective() |
| 150 |
fit_objective.addFitPair(simulation_builder, data) |
| 151 |
``` |
| 152 |
|
| 153 |
The array must match the detector pixel layout defined in the |
| 154 |
simulation: same number of rows and columns. |
| 155 |
|
| 156 |
[galaxi]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/scatter2d/expfit_galaxi.py |
| 157 |
[fitgisas]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/scatter2d/fit_gisas.py |
| 158 |
[spec1par]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Specular1Par.py |
| 159 |
[honeycomb]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Honeycomb_fit.py |
| 160 |
[ptlayer]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Pt_layer_fit.py |
| 161 |
[treff]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/TREFF_Ni_film.py |