Debian Codegraph @

hugo/content/ref/fit/data-import/_index.md

raw
1 +++
2 title = "Data import"
3 weight = 10
4 +++
5
6 ## Overview
7
8 BornAgain provides functions to read 1D and 2D data fields:
9
10 ```python
11 ba.readData1D(fname)
12 ba.readData1D(fname, ftype)
13 ba.readData1D(fname, ftype, import_settings)
14
15 ba.readData2D(fname)
16 ba.readData2D(fname, ftype)
17 ba.readData2D(fname, ftype, import_settings)
18 ```
19
20 The single-argument forms determine the file format from the file name
21 extension (see tables below).
22 If the name ends with `.gz` or `.bz2`, the file is decompressed on the fly.
23
24 The multi-argument forms accept an explicit format constant (`ftype`),
25 needed when the file extension does not match the format or is not specific enough.
26 A third, optional argument is an `ImportSettings` object
27 that specifies in particular the column layout.
28 It is always required for 1D CSV files.
29
30 ## Supported formats
31
32 ##### 1D formats (for specular reflectometry)
33
34 | Format | Extension | Enum constant |
35 |--------|-----------|---------------|
36 | BornAgain internal text format | `.int` | `ba.bornagain1D` |
37 | Motofit format | `.mft` | `ba.mft` |
38 | CSV/ASCII column data | any other | `ba.csv1D` |
39
40 ##### 2D formats (for GISAS/diffraction)
41
42 | Format | Extension | Enum constant |
43 |--------|-----------|---------------|
44 | BornAgain internal text format | `.int` | `ba.bornagain2D` |
45 | [TIFF image](tiff) | `.tif`, `.tiff` | `ba.tiff` |
46 | BERsans format | `.001` | `ba.nicos2D` |
47 | CSV/ASCII matrix data | any other | `ba.csv2D` |
48
49 ## Reading 1D data
50
51 For CSV/ASCII column files, `ImportSettings1D` is always required
52 (the single-argument form throws for CSV, as it cannot infer column layout).
53 Pass format and settings explicitly:
54
55 ```python
56 flags = ba.ImportSettings1D("2alpha (deg)", "#", "", 1, 2)
57 data = ba.readData1D(fname, ba.csv1D, flags)
58 ```
59
60 For self-describing formats (`.int`, `.mft`), no settings are needed:
61
62 ```python
63 data = ba.readData1D("reflectivity.mft")
64 ```
65
66 **Examples:**
67 [Specular1Par.py][spec1par],
68 [Honeycomb_fit.py][honeycomb],
69 [Pt_layer_fit.py][ptlayer],
70 [TREFF_Ni_film.py][treff].
71
72 ##### ImportSettings1D
73
74 `ImportSettings1D` describes the column layout of a CSV reflectivity file.
75 Column indices count from 1.
76
77 ```python
78 flags = ba.ImportSettings1D(
79 xCoord, # coordinate type, e.g. "q_z (1/nm)" or "2alpha (deg)"
80 headerPrefix, # prefix that marks header lines, e.g. "#"
81 linesToSkip, # line-number pattern to skip, e.g. "1,10-12"
82 col_Q, # column number of Q (or angle)
83 col_R, # column number of R (intensity)
84 col_sR=0, # column of sigma_R (uncertainty), 0 = absent
85 col_dQ=0, # column of delta_Q (resolution), 0 = absent
86 col_lambda=0, # column of wavelength, 0 = absent
87 sort=True, # sort rows by Q
88 rm_negative=True, # discard rows with negative Q
89 rm_duplications=True # discard rows with duplicate Q
90 )
91 ```
92
93 ## Reading 2D data
94
95 The simplest call uses automatic format detection from the file name:
96
97 ```python
98 data = ba.readData2D("detector_image.tif.gz")
99 ```
100
101 An explicit file type can be given as second argument.
102 For CSV format, `ImportSettings2D` is also required:
103
104 ```python
105 data = ba.readData2D(fname, ba.csv2D, import_settings)
106 ```
107
108 **Examples:**
109 [expfit_galaxi.py][galaxi] loads a TIFF file and fits a GISAS pattern
110 from the GALAXI instrument.
111 [fit_gisas.py][fitgisas] loads a BornAgain internal-format file.
112
113 ##### ImportSettings2D
114
115 When reading a 2D CSV file, `ImportSettings2D` controls how axes are parsed:
116
117 ```python
118 settings = ba.ImportSettings2D(
119 xCoord, # coordinate type of the x axis, e.g. "phi_f (deg)"
120 yCoord, # coordinate type of the y axis, e.g. "alpha_f (deg)"
121 has_axes=False, # True if the file contains explicit axis values
122 swap_axes=False, # True to swap x and y
123 first_row=True, # True if axis values are in the first row (else last)
124 first_col=True, # True if axis values are in the first column (else last)
125 )
126 ```
127
128 ## Reading using fabio and NumPy
129
130 If your instrument produces a format not listed above, read the data
131 into a numpy array with a third-party library and wrap it in a
132 `Datafield` before passing it to `FitObjective`:
133
134 ```python
135 import fabio # example: EDF/CBF/HDF5 and many more
136 import bornagain as ba
137
138 img = fabio.open("experimental_data.edf")
139 arr = img.data.astype("float64")
140 nrows, ncols = arr.shape
141 frame = ba.Frame(ba.EquiDivision("x", ncols, 0.0, float(ncols)),
142 ba.EquiDivision("y", nrows, 0.0, float(nrows)))
143 data = ba.Datafield(frame, list(arr.flatten()))
144 ```
145
146 Then pass the `Datafield` to the fit engine:
147
148 ```python
149 fit_objective = ba.FitObjective()
150 fit_objective.addFitPair(simulation_builder, data)
151 ```
152
153 The array must match the detector pixel layout defined in the
154 simulation: same number of rows and columns.
155
156 [galaxi]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/scatter2d/expfit_galaxi.py
157 [fitgisas]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/scatter2d/fit_gisas.py
158 [spec1par]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Specular1Par.py
159 [honeycomb]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Honeycomb_fit.py
160 [ptlayer]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Pt_layer_fit.py
161 [treff]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/TREFF_Ni_film.py