+++ title = "Data import" weight = 10 +++ ## Overview BornAgain provides functions to read 1D and 2D data fields: ```python ba.readData1D(fname) ba.readData1D(fname, ftype) ba.readData1D(fname, ftype, import_settings) ba.readData2D(fname) ba.readData2D(fname, ftype) ba.readData2D(fname, ftype, import_settings) ``` The single-argument forms determine the file format from the file name extension (see tables below). If the name ends with `.gz` or `.bz2`, the file is decompressed on the fly. The multi-argument forms accept an explicit format constant (`ftype`), needed when the file extension does not match the format or is not specific enough. A third, optional argument is an `ImportSettings` object that specifies in particular the column layout. It is always required for 1D CSV files. ## Supported formats ##### 1D formats (for specular reflectometry) | Format | Extension | Enum constant | |--------|-----------|---------------| | BornAgain internal text format | `.int` | `ba.bornagain1D` | | Motofit format | `.mft` | `ba.mft` | | CSV/ASCII column data | any other | `ba.csv1D` | ##### 2D formats (for GISAS/diffraction) | Format | Extension | Enum constant | |--------|-----------|---------------| | BornAgain internal text format | `.int` | `ba.bornagain2D` | | [TIFF image](tiff) | `.tif`, `.tiff` | `ba.tiff` | | BERsans format | `.001` | `ba.nicos2D` | | CSV/ASCII matrix data | any other | `ba.csv2D` | ## Reading 1D data For CSV/ASCII column files, `ImportSettings1D` is always required (the single-argument form throws for CSV, as it cannot infer column layout). Pass format and settings explicitly: ```python flags = ba.ImportSettings1D("2alpha (deg)", "#", "", 1, 2) data = ba.readData1D(fname, ba.csv1D, flags) ``` For self-describing formats (`.int`, `.mft`), no settings are needed: ```python data = ba.readData1D("reflectivity.mft") ``` **Examples:** [Specular1Par.py][spec1par], [Honeycomb_fit.py][honeycomb], [Pt_layer_fit.py][ptlayer], [TREFF_Ni_film.py][treff]. ##### ImportSettings1D `ImportSettings1D` describes the column layout of a CSV reflectivity file. Column indices count from 1. ```python flags = ba.ImportSettings1D( xCoord, # coordinate type, e.g. "q_z (1/nm)" or "2alpha (deg)" headerPrefix, # prefix that marks header lines, e.g. "#" linesToSkip, # line-number pattern to skip, e.g. "1,10-12" col_Q, # column number of Q (or angle) col_R, # column number of R (intensity) col_sR=0, # column of sigma_R (uncertainty), 0 = absent col_dQ=0, # column of delta_Q (resolution), 0 = absent col_lambda=0, # column of wavelength, 0 = absent sort=True, # sort rows by Q rm_negative=True, # discard rows with negative Q rm_duplications=True # discard rows with duplicate Q ) ``` ## Reading 2D data The simplest call uses automatic format detection from the file name: ```python data = ba.readData2D("detector_image.tif.gz") ``` An explicit file type can be given as second argument. For CSV format, `ImportSettings2D` is also required: ```python data = ba.readData2D(fname, ba.csv2D, import_settings) ``` **Examples:** [expfit_galaxi.py][galaxi] loads a TIFF file and fits a GISAS pattern from the GALAXI instrument. [fit_gisas.py][fitgisas] loads a BornAgain internal-format file. ##### ImportSettings2D When reading a 2D CSV file, `ImportSettings2D` controls how axes are parsed: ```python settings = ba.ImportSettings2D( xCoord, # coordinate type of the x axis, e.g. "phi_f (deg)" yCoord, # coordinate type of the y axis, e.g. "alpha_f (deg)" has_axes=False, # True if the file contains explicit axis values swap_axes=False, # True to swap x and y first_row=True, # True if axis values are in the first row (else last) first_col=True, # True if axis values are in the first column (else last) ) ``` ## Reading using fabio and NumPy If your instrument produces a format not listed above, read the data into a numpy array with a third-party library and wrap it in a `Datafield` before passing it to `FitObjective`: ```python import fabio # example: EDF/CBF/HDF5 and many more import bornagain as ba img = fabio.open("experimental_data.edf") arr = img.data.astype("float64") nrows, ncols = arr.shape frame = ba.Frame(ba.EquiDivision("x", ncols, 0.0, float(ncols)), ba.EquiDivision("y", nrows, 0.0, float(nrows))) data = ba.Datafield(frame, list(arr.flatten())) ``` Then pass the `Datafield` to the fit engine: ```python fit_objective = ba.FitObjective() fit_objective.addFitPair(simulation_builder, data) ``` The array must match the detector pixel layout defined in the simulation: same number of rows and columns. [galaxi]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/scatter2d/expfit_galaxi.py [fitgisas]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/scatter2d/fit_gisas.py [spec1par]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Specular1Par.py [honeycomb]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Honeycomb_fit.py [ptlayer]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/Pt_layer_fit.py [treff]: https://jugit.fz-juelich.de/mlz/bornagain/-/blob/develop/rawEx/fit/specular/TREFF_Ni_film.py