| 1 |
1.30 2026-01-12 |
| 2 |
~~~~~~~~~~~~~~~ |
| 3 |
|
| 4 |
* Added support for Sentinel-4 L2. |
| 5 |
|
| 6 |
* Allow reading of .pth files in csv format containing additional columns. |
| 7 |
(located after those produced by 'harpdump --dataset'). |
| 8 |
|
| 9 |
* Updates to product ingestions: |
| 10 |
- GEOMS Pandora: added support for GEOMS-TE-PANDORA-OFFAXIS-GAS-001. |
| 11 |
- EarthCARE CPR_CLP_2A: renamed vertical_wind_speed variable to |
| 12 |
vertical_air_velocity. |
| 13 |
|
| 14 |
1.29 2025-11-28 |
| 15 |
~~~~~~~~~~~~~~~ |
| 16 |
|
| 17 |
* Added conversion to derive reflectance from radiance, solar irradiance |
| 18 |
and solar zenith angle. |
| 19 |
|
| 20 |
* Added HARP_UNIT_WAVELENGTH_RADIANCE and HARP_UNIT_WAVELENGTH_IRRADIANCE |
| 21 |
to harp.h. |
| 22 |
|
| 23 |
* Updates to product ingestions: |
| 24 |
- EarthCARE: added support for JAXA L1 and L2A products |
| 25 |
(CPR_NOM_1B, ATL_CLA_2A, CPR_ECO_2A, MSI_CLP_2A, CPR_CLP_2A). |
| 26 |
- EarthCARE L1 MSI: added wavelength_irradiance variable and renamed |
| 27 |
radiance to wavelength_radiance. |
| 28 |
|
| 29 |
1.28.1 2025-11-13 |
| 30 |
~~~~~~~~~~~~~~~~~ |
| 31 |
|
| 32 |
* Fixed regression in HARP 1.25 where performing a point or polygon filter |
| 33 |
during ingestion would result in a segmentation fault if the latitude / |
| 34 |
longitude variables contained single precision floating point values. |
| 35 |
|
| 36 |
1.28 2025-11-10 |
| 37 |
~~~~~~~~~~~~~~~ |
| 38 |
|
| 39 |
* Updates to product ingestions: |
| 40 |
- IASI-NG L2 CLD: added cloud property variables. |
| 41 |
- IASI-NG L2 SFC: renamed dust variable to dust_aerosol_index. |
| 42 |
- IASI-NG L2 SO2: added SO2 column and layer height variables. |
| 43 |
- IASI-NG L2 TCWV: added missing water_vapor_column_density. |
| 44 |
- S5 L1B: updated description clarifying that wavelength values are |
| 45 |
based on evaluating Chebyshev coefficients. |
| 46 |
- S5P L1b: error/noise values of -126 are now mapped to NaN. |
| 47 |
- S5P L2 HCHO/SO2: added scene_albedo and scene_pressure, removed amf |
| 48 |
precission, and added snow_ice_type and sea_ice_fraction. |
| 49 |
- S5P L2 SO2: added 3km so2 type. |
| 50 |
|
| 51 |
1.27 2025-10-13 |
| 52 |
~~~~~~~~~~~~~~~ |
| 53 |
|
| 54 |
* Added support for Sentinel-5 L1B and IASI-NG L2. |
| 55 |
|
| 56 |
* Improve performance of derive() operations to compensate for the |
| 57 |
regression introduced in HARP 1.26. |
| 58 |
|
| 59 |
* harpdump --dataset and .pth files can now contain string values between |
| 60 |
double quotes to store filenames with a comma in them (which is applicable |
| 61 |
for MTG and EPS-SG products such as Sentinel-4 and Sentinel-5). |
| 62 |
|
| 63 |
* Updates to product ingestions: |
| 64 |
- CLOUDNET L2 classification: map time series of sensor position to scalar. |
| 65 |
- GEOMS SONDE: added support for GEOMS-TE-SONDE-O3-003. |
| 66 |
- GOSAT FTS L2: added XCH4/XCO2 variables to ingestion. |
| 67 |
- S5P L2: added land_fraction variable for AAI/ALH/CH4/CO/FRESCO/NO2/O3PR. |
| 68 |
- S5P L2 AER_AI: added s354_388nm_scm option + cloud/albedo variables. |
| 69 |
- S5P L2 FRESCO: added scene_height and land_fraction variables. |
| 70 |
- Sentinel-5 L2: fixed structure and values of datetime variables. |
| 71 |
|
| 72 |
1.26 2025-07-09 |
| 73 |
~~~~~~~~~~~~~~~ |
| 74 |
|
| 75 |
* Added support for Sentinel-5 L2. |
| 76 |
|
| 77 |
* Added derivations to retrieve (dry) air column number density from |
| 78 |
species column number density and species (dry air) volume mixing ratio. |
| 79 |
|
| 80 |
* Added derivations to treat vertical profiles of volume mixing ratios |
| 81 |
and partial column profiles of column volume mixing ratios as equal. |
| 82 |
|
| 83 |
* Added 'invalid()' operation to allow filtering on invalid values. |
| 84 |
|
| 85 |
* Fixed memory corruption issue in rebinning algorithm. |
| 86 |
|
| 87 |
* Fixed issue where 'index(dimension)' operation would not get applied |
| 88 |
correctly when reading data via an ingestion module, the dimension for |
| 89 |
the index filter was not time, and it was combined with another filter. |
| 90 |
|
| 91 |
* Updates to product ingestions: |
| 92 |
- EarthCARE L1: fixed TIR band ingestion options for MSI_NOM_1B. |
| 93 |
- EarthCARE L2: fixed typo in variable name for MSI_AOT_2A. |
| 94 |
- S5P L1B: added systematic and random uncertainty variables. |
| 95 |
- S5P L2 O3 TCL: fixed unit of mixing ratios (ppbv -> ppv). |
| 96 |
|
| 97 |
1.25 2025-03-16 |
| 98 |
~~~~~~~~~~~~~~~ |
| 99 |
|
| 100 |
* The rebin() operation now takes into account existing weight variables |
| 101 |
and properly handles angle variables. Rebinning the result of a |
| 102 |
bin_spatial() operation to a lower resolution now gives the same result |
| 103 |
as when a spatial bin was performed directly to that lower resolution. |
| 104 |
|
| 105 |
* Added harp_convert_unit_double() and harp_convert_unit_float(). |
| 106 |
harp_convert_unit() is now deprecated. |
| 107 |
|
| 108 |
* Performing a unit conversion on single precision floating point |
| 109 |
variables will no longer convert the variable to double precision. |
| 110 |
|
| 111 |
* Updates to product ingestions: |
| 112 |
- GOME2 L2: added support for NHP/OHP ozone profile products and all |
| 113 |
products now use float instead of double for variables. |
| 114 |
- GEOMS FTIR: reinterpret vmr from old GEOMS templates as dry air vmr. |
| 115 |
- EarthCARE L1: add support for L1 ATLID, BBR, and MSI products. |
| 116 |
- EarthCARE L2: all vertical profiles are inverted to make them |
| 117 |
ascending, fixed geoid offset correction, and fixed AM__CTH_2B |
| 118 |
ingestion. |
| 119 |
- S5P PAL L2: added support for Ocean Colour (L2__KD____) product. |
| 120 |
|
| 121 |
1.24 2024-11-11 |
| 122 |
~~~~~~~~~~~~~~~ |
| 123 |
|
| 124 |
* Fixed appending non-float variables with different sub-dimensions when |
| 125 |
using the Python harp.concatenate() function. |
| 126 |
|
| 127 |
* Updates to product ingestions: |
| 128 |
- ESACCI_GHG: fixed incorrect unit for CH4/CO variables |
| 129 |
- GOME-2 L2: added support for O3MARS aerosol layer height and added |
| 130 |
support for O3MARP V5 |
| 131 |
- S5P L2: added snow_ice_type/sea_ice_fraction ingestion for AAI/CO/CH4 |
| 132 |
- S5P L2 CH4: added 'corrected' option to ingest destriped columns and |
| 133 |
fixed issue in NIR/SWIR selection of surface albedo |
| 134 |
- S5P PAL L2: added support for SRON/SICOR HDO (L2__HDO__S) product |
| 135 |
|
| 136 |
1.23 2024-07-30 |
| 137 |
~~~~~~~~~~~~~~~ |
| 138 |
|
| 139 |
* Fixed numpy 2.x compatibility issues in python interface. |
| 140 |
|
| 141 |
* Added derivations to retrieve dry air mass/volume mixing ratio from H2O |
| 142 |
mass/volume mixing ratio with regard to dry air. |
| 143 |
|
| 144 |
* Fixed some issues with building the HARP R interface extension. |
| 145 |
|
| 146 |
* Updates to product ingestions: |
| 147 |
- S5P PAL L2 AOT: added support for cloud_fraction vs. |
| 148 |
effective_cloud_fraction split |
| 149 |
|
| 150 |
1.22 2024-06-12 |
| 151 |
~~~~~~~~~~~~~~~ |
| 152 |
|
| 153 |
* Updates to product ingestions: |
| 154 |
- EARLINET: fixed memory issue when reading extinction data. |
| 155 |
- EarthCARE: updates for new V11.2 format definition baseline |
| 156 |
- GEOMS UVVIS.DOBSON: fixed issue with reading of int16 typed data. |
| 157 |
- S5P PAL L2 SO2 COBRA: fixed cloud_fraction option behaviour and added |
| 158 |
alternative qa value calculation for processor V02.00.00. |
| 159 |
|
| 160 |
1.21 2024-03-15 |
| 161 |
~~~~~~~~~~~~~~~ |
| 162 |
|
| 163 |
* Fixed incorrect weight division for random uncertainties in spatial binning. |
| 164 |
|
| 165 |
* Added check for unit equality when appending variables. |
| 166 |
|
| 167 |
* Fixed CMake build for locating Python 3.12. |
| 168 |
|
| 169 |
* Updates to product ingestions: |
| 170 |
- EarthCARE: updates for new V11 format definition baseline |
| 171 |
- ECMWF GRIB: added general support for new masterTablesVersion numbers |
| 172 |
- S5P L2 CH4: added support for nir aerosol/cloud/surface property ingestion. |
| 173 |
- S5P L2 CLOUD: added support for nir-band spatial grid data |
| 174 |
- S5P PAL L2: added support for OClO |
| 175 |
- S5P PAL L2 AOT: added support for V02.00.00 output format |
| 176 |
- S5P PAL L2 BRO: fixed memory issue with reading total column and |
| 177 |
uncertainty variables |
| 178 |
- S5P PAL L2 TCWV: fixed wrong quantity for density variables |
| 179 |
|
| 180 |
1.20.2 2023-11-13 |
| 181 |
~~~~~~~~~~~~~~~~~ |
| 182 |
|
| 183 |
* Fill values of floating point types are now explicitly set to NaN when |
| 184 |
exporting to a netcdf or hdf5 file. |
| 185 |
|
| 186 |
* Updates to product ingestions: |
| 187 |
- EARLINET: fixed issue where no backscatter/extinction data was ingested. |
| 188 |
|
| 189 |
1.20.1 2023-10-12 |
| 190 |
~~~~~~~~~~~~~~~~~ |
| 191 |
|
| 192 |
* Updates to product ingestions: |
| 193 |
- IASI L1C: added missing angle variables |
| 194 |
- S5P PAL L2 AER_OT: added missing wavelength variable |
| 195 |
- S5P PAL L2 CHOCHO: fixed missing units for number density variables |
| 196 |
|
| 197 |
1.20 2023-09-19 |
| 198 |
~~~~~~~~~~~~~~~ |
| 199 |
|
| 200 |
* Updates to product ingestions: |
| 201 |
- GEOMS FTIR: added support for C2H3NO5/PAN and C2H4, added support for |
| 202 |
template V3. |
| 203 |
- GEOMS TCCON: added support for template V6. |
| 204 |
- S5P L2 ALH: added support for revised surface_albedo variable and added |
| 205 |
surface_albedo_uncertainty. |
| 206 |
- S5P L2 O3: fixed custom V1 QA filter for NRTI products |
| 207 |
- S5P PAL L2: fixed memory issue when reading aerosol_optical_depth_validity |
| 208 |
of AER_OT product. |
| 209 |
|
| 210 |
* Fixed missing documentation on weight factor handling in binning algorithm. |
| 211 |
|
| 212 |
* Updated embedded expat library to version 2.5.0. |
| 213 |
|
| 214 |
* Fixed issue where jpeg dll was missing from windows binary installer. |
| 215 |
|
| 216 |
1.19 2023-07-24 |
| 217 |
~~~~~~~~~~~~~~~ |
| 218 |
|
| 219 |
* The python harp.import_product() function now allows HARP .pth files to be |
| 220 |
provided as input. |
| 221 |
|
| 222 |
* Reintroduced propagation of *_uncertainty_random variables when performing |
| 223 |
spatial binning with area weighted averaging. |
| 224 |
|
| 225 |
* Allowed regridding using a multi-dimensional source axis-variable as input. |
| 226 |
e.g. convert from an 'altitude {latitude,longitude,vertical}' grid to |
| 227 |
a fixed 'altitude {vertical}` grid. |
| 228 |
|
| 229 |
* Added documentation for regridding/binning algorithms. |
| 230 |
|
| 231 |
* Updates to product ingestions: |
| 232 |
- ACE-FTS: fix ingestion of ASCII formatted products |
| 233 |
- GOME-2 L2: added support for offline absorbing aerosol index (O3MARP) |
| 234 |
- EARLINET: added support for new format (removed support for old format) |
| 235 |
- EarthCARE: added support for AM__ACD_2B, AM__CTH_2B, BM__RAD_2B, |
| 236 |
BMA_FLX_2B, MSI_AOT_2A, MSI_CM__2A, and MSI_COP_2A |
| 237 |
- ECMWF GRIB: added support for new masterTablesVersion=30 data |
| 238 |
- S5P PAL L2: added support for AER_OT, BRO, CHOCHO, SIF, SO2 COBRA, and |
| 239 |
TCWV |
| 240 |
|
| 241 |
1.18 2023-06-08 |
| 242 |
~~~~~~~~~~~~~~~ |
| 243 |
|
| 244 |
* Renamed all uses of 'water vapour' to 'water vapor'. |
| 245 |
|
| 246 |
* Added =| bitfield operater to allow filtering for the case where only some |
| 247 |
bits within the mask are set. |
| 248 |
|
| 249 |
* The uncertainty propagation of *_uncertainty_random variables for the |
| 250 |
temporal binning operation now always assumes no correlation. For spatial |
| 251 |
binning the random component is removed for the area weighted average |
| 252 |
approach and it is propagated using no correlation for the simple binning |
| 253 |
based on center points. |
| 254 |
|
| 255 |
* Added derivation for 'area {latitude,longitude}' and fixed area calculation |
| 256 |
where South Pole was an edge point. |
| 257 |
|
| 258 |
* Fixed reading of string arrays in HARP Python interface. |
| 259 |
|
| 260 |
* Updates to product ingestions: |
| 261 |
- ECMWF GRIB: added support for new masterTablesVersion=29 data |
| 262 |
- S5P L2 CLOUD: fixed qa_value ingestion for CRB (now use qa_value_crb) |
| 263 |
- S5P L2 SO2: added support for SO2 layer height |
| 264 |
- OMI L3: added support for OMHCHOd, OMNO2d, and OMSO2e |
| 265 |
|
| 266 |
1.17 2022-12-12 |
| 267 |
~~~~~~~~~~~~~~~ |
| 268 |
|
| 269 |
* Added additional conventions for variable names related to areosol/cloud |
| 270 |
properties. |
| 271 |
|
| 272 |
* Fixed issue with geographical area overlap calculation and improvided |
| 273 |
performance of performing operations on geographical polygon data. |
| 274 |
|
| 275 |
* Windows .msi installer now allows HARP component selection for core, Python, |
| 276 |
IDL, MATLAB, and R. |
| 277 |
|
| 278 |
* Updates to product ingestions: |
| 279 |
- AEOLUS L2B: properly split datetime, datetime_start, and datetime_stop |
| 280 |
- CCI L3 O3 LP: fixed wrong variable name |
| 281 |
- EarthCARE: added additional product types and updated ACM_CAP_2B |
| 282 |
- ECMWF GRIB: changed RWC/SWC/LWC/IWC variable names |
| 283 |
- OSIRIS L2: fixed wrong variable names |
| 284 |
- TES TL2ATMTN/TL2ATMTL: fixed wrong variable names |
| 285 |
|
| 286 |
1.16 2022-10-31 |
| 287 |
~~~~~~~~~~~~~~~ |
| 288 |
|
| 289 |
* Improved algorithm to calculate center point from latitude/longitude bounds |
| 290 |
polygon. |
| 291 |
|
| 292 |
* Added derivation of time dependent from time indepent for datetime variables. |
| 293 |
|
| 294 |
* Updates to product ingestions: |
| 295 |
- GEOMS PANDORA: added systematic/random uncertainty for effective |
| 296 |
temperature, renamed H2CO in harp variable names to HCHO. |
| 297 |
- OCO: added support for ACOS/OCO-2/OCO-3 L2/L3 products. |
| 298 |
- Ozone CCI L3 LP: fixed memory issue when reading |
| 299 |
merged_ozone_concentration. |
| 300 |
- S5P L2 HCHO: added amf=clear_sky ingestion option |
| 301 |
|
| 302 |
1.15.1 2022-05-18 |
| 303 |
~~~~~~~~~~~~~~~~~ |
| 304 |
|
| 305 |
* Updates to product ingestions: |
| 306 |
- CLOUDNET L2 classification: fixed fillvalue mapping of values to NaN. |
| 307 |
- GEOMS PANDORA: fixes for template V003 support |
| 308 |
- GEOMS FTIR: allow instrument lat/lon/alt variables to be one-dimensional |
| 309 |
with replicated values. |
| 310 |
- GEOMS UVVIS.BREWER: added random+systematic uncertainty variables. |
| 311 |
- S5P L2 CO: fixed vertical ordering of new apriori profile. |
| 312 |
|
| 313 |
1.15 2022-03-14 |
| 314 |
~~~~~~~~~~~~~~~ |
| 315 |
|
| 316 |
* Fixed issue where bin() operation was removing scalar enumeration variables. |
| 317 |
|
| 318 |
* Updates to product ingestions: |
| 319 |
- ECMWF GRIB: fixed parameter mapping for CAMS model data that uses |
| 320 |
masterTablesVersion=27 data and added support for new |
| 321 |
masterTablesVersion=28 data |
| 322 |
- GEOMS UVVIS.BREWER: added support for template V001 |
| 323 |
- GEOMS PANDORA: added support for template V003 |
| 324 |
- S5P L2 AAI: added new 335/367 wavelength pair |
| 325 |
- S5P L2 CO: added new apriori profile and support for new AVK unit |
| 326 |
- S5P L2 O3 Profile: added surface_albedo and cloud_albedo |
| 327 |
- S5P L2 HCHO/O3/SO2: added support for updated NRTI indication attribute. |
| 328 |
|
| 329 |
1.14 2021-10-20 |
| 330 |
~~~~~~~~~~~~~~~ |
| 331 |
|
| 332 |
* Updated embedded udunits2 library to version 2.2.28. |
| 333 |
|
| 334 |
* Improved CMake build system. |
| 335 |
|
| 336 |
* Fixed issue when building against HDF5 1.12. |
| 337 |
|
| 338 |
* Updates to product ingestions: |
| 339 |
- AEOLUS L1B: fixed ingestion of Mie/measurement data; added sensor angles |
| 340 |
- AEOLUS L2B: fixed ingestion of Mie data; added sensor angles |
| 341 |
- CLOUDNET L2 classification: added support for new (netcdf4) format |
| 342 |
- EARTHCARE: added very early support (limited version of ECA_ACM_CAP_2B) |
| 343 |
- ECMWF GRIB: fixed unit of backscatter coefficient 1/(milli sr) -> 1/(m sr), |
| 344 |
added support for new masterTablesVersion=27 data |
| 345 |
- GEOMS UVVIS.DOAS: allow template V007 files to be stored in hdf5 format |
| 346 |
- GHG CCI: added support for TROPOMI WFMD L2 |
| 347 |
- OMI L2: added sensor alt/lat/lon and surface altitude/pressure |
| 348 |
- OMI L2 OMAERUV: fix ingestion issues |
| 349 |
- QA4ECV HCHO: added amf=clear_sky ingestion option |
| 350 |
- S5P L2 HCHO: renamed cloud_altitude variable to cloud_height |
| 351 |
- S5P L2 NO2: added retrieval of O2-O2 cloud (O22CLD) data |
| 352 |
- S5P L2 NO2+FRESCO: added scene_albedo and scene_pressure |
| 353 |
- S5P L2 O3 Profile: fixed wrong concentration variables (vmr -> nd) |
| 354 |
- S5P L2 SO2: updated SO2_type enumeration list |
| 355 |
|
| 356 |
1.13 2021-03-12 |
| 357 |
~~~~~~~~~~~~~~~ |
| 358 |
|
| 359 |
* Added rebin() operation. |
| 360 |
|
| 361 |
* Added harp_product_rebin_with_axis_bounds_variable() function. |
| 362 |
|
| 363 |
* Added support for datetime_bounds variable. |
| 364 |
|
| 365 |
* Fixed issue in =& operation (it returned true if any of the bits were 1 |
| 366 |
instead of all bits). |
| 367 |
|
| 368 |
* Updates to product ingestions: |
| 369 |
- GEOMS: renamed all site_name variables to location_name |
| 370 |
- GEOMS FTIR H2CO: fixed issue in determining unit for H2O data (that would |
| 371 |
fail the ingestion) |
| 372 |
- S5P L2 CO: added co=corrected ingestion option |
| 373 |
- SCIAMACHY L2: fixed averaging of cloud fraction for co-added pixels |
| 374 |
|
| 375 |
1.12 2020-10-29 |
| 376 |
~~~~~~~~~~~~~~~ |
| 377 |
|
| 378 |
* The csv output of 'harpdump --dataset' can now be used as content of a .pth |
| 379 |
file. This improves the performance of harpcollocate/harpmerge/etc. as they |
| 380 |
no longer need to open and extract metadata from files themselves anymore. |
| 381 |
|
| 382 |
* harpcollocate now stores differences as signed values in collocation result |
| 383 |
file (instead of absolute differences). |
| 384 |
|
| 385 |
* Added harp_dataset_prefilter(). This function is now also called within |
| 386 |
harpmerge and harpcollocate to pre-filter the input dataset(s) for datetime |
| 387 |
and collocate left/right filters that are at the start of the operations |
| 388 |
string. |
| 389 |
|
| 390 |
* Added format and history fields to harp_product_metadata. |
| 391 |
|
| 392 |
* Added harp.import_product_metadata() to Python interface. |
| 393 |
|
| 394 |
* harp_dataset_add_product() now ignores files/directories starting with '.'. |
| 395 |
|
| 396 |
* Added uncertainty propagation to bin() and bin_spatial() operations |
| 397 |
(using uncorrelated assumption for bin() and correlated for bin_spatial()). |
| 398 |
|
| 399 |
* Added global option to determine whether uncertainty propagation should |
| 400 |
use a correlated or uncorrelated approach. Currently only used for the bin() |
| 401 |
operation. |
| 402 |
|
| 403 |
* Added global option to create a collocation_datetime variable when performing |
| 404 |
a collocate_left() or collocate_right() operation. |
| 405 |
|
| 406 |
* Added wildcard pattern matching ('*'/'?') to keep() and exclude() operations. |
| 407 |
|
| 408 |
* Added derivation to retrieve angstrom exponent from aerosol optical depth. |
| 409 |
|
| 410 |
* Added derivation to retrieve tropospheric/stratospheric column avk from |
| 411 |
total column avk and tropopause altitude. |
| 412 |
|
| 413 |
* Fixed potential memory issues and improved overall code stability. |
| 414 |
|
| 415 |
* Updates to product ingestions: |
| 416 |
- S5P L2 CLOUD: added cloud_type and cloud_top_temperature |
| 417 |
- S5P L2 O3: added qa_filter=custom ingestion option to set validity based |
| 418 |
on official PRF recommendation for processor V1.x generated products |
| 419 |
- MLS L2: fixed memory issue in reading of validity variable and updated |
| 420 |
validity flag mapping to version 4.x-4.0 of the Data Quality document |
| 421 |
|
| 422 |
1.11 2020-05-19 |
| 423 |
~~~~~~~~~~~~~~~ |
| 424 |
|
| 425 |
* Added derivations to retrieve column number density from column vmr. |
| 426 |
|
| 427 |
* Fixed issue in bin_spatial() where the weightfactor could be too high when |
| 428 |
oversampling the input data. Grid cells that intersected with the data |
| 429 |
polygon could end up with a weightfactor >1.0 (but <2.0). |
| 430 |
|
| 431 |
* Fixed issue where molar mass was incorrectly provided for the species with |
| 432 |
the shortest matching name. e.g. the molar mass of CO was returned for CO2 |
| 433 |
and of NO for NO2. This impacts vmr <-> mmr conversions for BrO2, COF2, COS, |
| 434 |
CO2, C2H2O2, H2O2, NOCl, NO2, N2O, N2O5, and all isotopologues (e.g. H2O_161). |
| 435 |
|
| 436 |
* Removed irrelevant harp_str64() and harp_str64u() functions (HARP does not |
| 437 |
support 64bit integers). |
| 438 |
|
| 439 |
* Fixed issue where molar mass linked to variables with a 'surface', |
| 440 |
'tropospheric', or 'stratospheric' prefix returned 0. |
| 441 |
|
| 442 |
* Fixed error/crash in deriving dry air column vmr from dry air column mmr. |
| 443 |
|
| 444 |
* Updates to product ingestions: |
| 445 |
- GOME-2 L2: added support for SO2 columns at all plume heights |
| 446 |
- OMI L2 CLDO2/CLDRR: added validity variable |
| 447 |
|
| 448 |
1.10 2020-03-31 |
| 449 |
~~~~~~~~~~~~~~~ |
| 450 |
|
| 451 |
* Added support for -ar/--reduce-operations parameter in harpmerge and |
| 452 |
for reduce_operations parameter in harp.import_product() Python function. |
| 453 |
|
| 454 |
* Fixed incorrect "invalid variable" error on weight variables for angular |
| 455 |
quantities when performing a harpmerge. |
| 456 |
|
| 457 |
* Fixed issue where the udunits2 library could not be initialized if the |
| 458 |
environment had a LC_NUMERIC locale that did not have '.' as decimal |
| 459 |
separator. |
| 460 |
|
| 461 |
* Fixed issue in name mangling of embedded expat library. |
| 462 |
|
| 463 |
* Updates to product ingestions: |
| 464 |
- GOMOS L1 Limb: fixed unit of wavelength_photon_radiance (and uncertainty) |
| 465 |
and documented conversion algorithm |
| 466 |
- S5P L2 O3 PR: support for IODD 12.0.0 |
| 467 |
- S5P L2 O3 TPR: removed support (future availability of product uncertain) |
| 468 |
- GEOMS PANDORA: allow for trailing \0 in global attribute values |
| 469 |
|
| 470 |
1.9.2 2019-12-01 |
| 471 |
~~~~~~~~~~~~~~~~ |
| 472 |
|
| 473 |
* Fixed issue where using the 'bin()' operation did not correctly create a |
| 474 |
'<variable>_weight' variable from a 'weight' variable when it was missing. |
| 475 |
|
| 476 |
* Added default units to documention on variable name conventions. |
| 477 |
|
| 478 |
* Updates to product ingestions: |
| 479 |
- Calipso L2: fixed unit of backscatter coefficient |
| 480 |
- ERS GOME L2: fixed unit of cloud top height |
| 481 |
- S5P L2 AER_LH: fixed unit of aerosol optical depth |
| 482 |
- S5P L2 SO2: add scaled avk for 1km/7km/15km box profiles |
| 483 |
|
| 484 |
1.9.1 2019-11-01 |
| 485 |
~~~~~~~~~~~~~~~~ |
| 486 |
|
| 487 |
* Fixed issue where results from deriving partial column densities from vmr and |
| 488 |
pressure were a factor 10 too low. |
| 489 |
|
| 490 |
* Fixed issue where operation following an index filter operation did not |
| 491 |
get executed. |
| 492 |
|
| 493 |
1.9 2019-10-21 |
| 494 |
~~~~~~~~~~~~~~ |
| 495 |
|
| 496 |
* Added operations to filter on dimension indices (e.g. 'index(vertical) < 5'). |
| 497 |
|
| 498 |
* Changed bin() and bin_spatial() operations to use/produce separate 'count' |
| 499 |
and 'weight' variables. The counts/weights are used to allow the following |
| 500 |
commands to now be equivalent: |
| 501 |
harpmerge -a 'bin()' -ap 'bin()' a.nc b.nc merged.nc |
| 502 |
harpmerge -ap 'bin()' a.nc b.nc merged.nc |
| 503 |
As well as the following: |
| 504 |
harpmerge -a 'bin_spatial(...)' |
| 505 |
-ap 'squash(time, (latitude_bounds, longitude_bounds));bin()' |
| 506 |
a.nc b.nc merged.nc |
| 507 |
harpmerge -ap 'bin_spatial(...)' a.nc b.nc merged.nc |
| 508 |
|
| 509 |
* Fixed issue with invalid bounding box optimization for spherical area |
| 510 |
filter operations. |
| 511 |
|
| 512 |
* Fixed missing error handling when using invalid operations in |
| 513 |
harp.export_product() in Python. |
| 514 |
|
| 515 |
* Fixed Python3 compatibility issue when exporting string data. |
| 516 |
|
| 517 |
* Updates to ECMWF GRIB, IASI L1/L2, GOME-2 L1/L2, MIPAS L1, and S5P L2 |
| 518 |
product ingestions. |
| 519 |
|
| 520 |
1.8 2019-07-31 |
| 521 |
~~~~~~~~~~~~~~ |
| 522 |
|
| 523 |
* Added R interface. |
| 524 |
|
| 525 |
* Improved derivation for tropopause altitude/pressure (to not return NaN). |
| 526 |
|
| 527 |
* Added derivations for gravity. |
| 528 |
|
| 529 |
* Added derivations to convert between total air and dry air mixing ratios. |
| 530 |
|
| 531 |
* Fixed valid() operation to properly use valid_min/valid_max attribute values. |
| 532 |
|
| 533 |
* Fixed issue in adding *_count variable in append/merge when count variable |
| 534 |
had more than one dimension. |
| 535 |
|
| 536 |
* Updates to ECMWF GRIB, GEOMS MWR, and S5P L2 product ingestions. |
| 537 |
|
| 538 |
1.7 2019-06-06 |
| 539 |
~~~~~~~~~~~~~~ |
| 540 |
|
| 541 |
* Added --no-history option to harpconvert, hardump, and harpmerge. |
| 542 |
|
| 543 |
* Added harp.execute_operations() and harp.convert_unit() to Python interface. |
| 544 |
|
| 545 |
* Added optional min_collocation_index/max_collocation_index arguments to |
| 546 |
collocate_left()/collocate_right() operations. |
| 547 |
|
| 548 |
* harp_product_set_history() and harp_product_set_source_product() now accept |
| 549 |
NULL as argument to clear the current value. |
| 550 |
|
| 551 |
* Appending/merging products now automatically adds missing *_count variables. |
| 552 |
|
| 553 |
* Fixed potential crash during initialization of HARP ingestion module. |
| 554 |
|
| 555 |
* Fixed invalid results when using bin_spatial with latitude_bounds {time,2} |
| 556 |
and longitude_bounds {time,2} variables (i.e. range instead of polygon). |
| 557 |
|
| 558 |
* Fixed array ordering in MATLAB interface. Multi-dimensional arrays are now |
| 559 |
represented from fastest running to slowest (=fortran ordering) as intended. |
| 560 |
|
| 561 |
* Update to S5P L2 product ingestion. |
| 562 |
|
| 563 |
1.6 2019-04-29 |
| 564 |
~~~~~~~~~~~~~~ |
| 565 |
|
| 566 |
* Added clamp() and squash() operations. |
| 567 |
|
| 568 |
* Allow bin() and sort() operations to be performed using a set of variables. |
| 569 |
|
| 570 |
* harp_product_sort() now takes a list of variables. |
| 571 |
|
| 572 |
* The rename() operation no longer fails if the source variable does not exist |
| 573 |
but a variable with the target name already exists. |
| 574 |
|
| 575 |
* The python harp.import_product() function now uses the C library to perform |
| 576 |
the contactenation of products. |
| 577 |
|
| 578 |
* Added post_operations argument to python harp.import_product() function. |
| 579 |
|
| 580 |
* Besides using a polygon of >=3 vertices, a geospatial area can now also be |
| 581 |
defined via a bounding rect. Use e.g. 'latitude_bounds {time,2}' and |
| 582 |
'longitude_bounds {time,2}' to define the min/max of the bounding rect. |
| 583 |
|
| 584 |
* Added derivation for tropospheric/stratospheric column from partial column |
| 585 |
profile and tropopause. |
| 586 |
|
| 587 |
* Corrected derivation for calculating tropopause altitude/pressure. |
| 588 |
|
| 589 |
* Deriving a variable will now use the conversion that requires the minimum |
| 590 |
amount of intermediate variables to be calculated. |
| 591 |
|
| 592 |
* Added support for scene albedo/pressure/type variables. |
| 593 |
|
| 594 |
* Fixed issue where regridding in the vertical dimension using pressure as axis |
| 595 |
could end up with wrong pressure_bounds values. |
| 596 |
|
| 597 |
* Fixed problem in matlab mex extension determination on linux when using the |
| 598 |
configure script. |
| 599 |
|
| 600 |
* Fixed setting of default CODA definition path and udunits2 xml path for |
| 601 |
Python on Windows. |
| 602 |
|
| 603 |
* Added format, variable count, and dimension sizes to harpcheck output. |
| 604 |
|
| 605 |
* Added support for TCCON products in GEOMS format. |
| 606 |
|
| 607 |
* Updates to AEOLUS L2A, ESACCI_OZONE_L2_TC, GEOMS FTIR, GOME2 L2, OMI L2 NO2, |
| 608 |
QA4ECV, and S5P L2 product ingestions. |
| 609 |
|
| 610 |
* Updated embedded expat library to version 2.2.6. |
| 611 |
|
| 612 |
1.5 2018-12-17 |
| 613 |
~~~~~~~~~~~~~~ |
| 614 |
|
| 615 |
* Fixed issue where vertical smoothing/regridding provided invalid results if |
| 616 |
input variables were available as float instead of double data. |
| 617 |
|
| 618 |
* harp_product_get_derived_variable() no longer converts data to double if |
| 619 |
no data type was provided and data already had the right unit. |
| 620 |
|
| 621 |
* Fixed issue where listing derivations for a specific variable would not show |
| 622 |
all possible conversions. |
| 623 |
|
| 624 |
* Add derivation for cloud height from cloud base/top height. |
| 625 |
|
| 626 |
* Added support for Dobson, Pandora and Sonde products in GEOMS format. |
| 627 |
|
| 628 |
* Updates to ACE-FTS, GOME/GOME-2 L2, and S5P L2 product ingestions. |
| 629 |
|
| 630 |
1.4 2018-09-28 |
| 631 |
~~~~~~~~~~~~~~ |
| 632 |
|
| 633 |
* Added -t/--target option to harpdump --list-derivations. |
| 634 |
|
| 635 |
* Added variable_name parameter to harp_doc_list_conversions(). |
| 636 |
|
| 637 |
* Added support for meridional/zonal wind speed, and planetary boundary layer |
| 638 |
height variables. |
| 639 |
|
| 640 |
* Added support for sic (Shannon information content) variables for vertical |
| 641 |
profiles. |
| 642 |
|
| 643 |
* Aerosol optical depth and extinction coefficient variables will now |
| 644 |
automatically use a log/log regridding in the spectral dimension. |
| 645 |
This is equivalent to using the Angstrom Exponent approach. |
| 646 |
|
| 647 |
* Fixed interpolation issue where points beyond the source grid could end up |
| 648 |
having an arbitrary wrong value (instead of NaN). This is a regression of |
| 649 |
harp 1.3. |
| 650 |
|
| 651 |
* Add derivations between wind speed/direction and zonal/meridional velocity. |
| 652 |
|
| 653 |
* Flattening a dimension now special cases a dimension length 1 (with less |
| 654 |
modifications to variables/product). |
| 655 |
|
| 656 |
* Fixed issue where a point distance of 0 could sometimes result in NaN. |
| 657 |
|
| 658 |
* Updates to S5P L2, GOME/GOME-2 L2, GEOMS UVVIS-DOAS-OFFAXIS, ECMWF GRIB, and |
| 659 |
AERONET product ingestions. |
| 660 |
|
| 661 |
1.3 2018-07-05 |
| 662 |
~~~~~~~~~~~~~~ |
| 663 |
|
| 664 |
* Added bin_spatial() operation. |
| 665 |
|
| 666 |
* Improved binning operation to automatically filter for NaN values and add |
| 667 |
variable-specific count variables to specify the number of non-NaN values. |
| 668 |
|
| 669 |
* Performing a binning operation on top of a binning operation will propagate |
| 670 |
averages based on the available count variables (i.e. as if the final bin was |
| 671 |
performed on the original data). |
| 672 |
|
| 673 |
* Entries added to the global 'history' attribute now also include the current |
| 674 |
utc time and harp version before the execution command. |
| 675 |
|
| 676 |
* Allow NULL as argument for harp_variable_set_description() and |
| 677 |
harp_variable_set_unit() to clear the current description/unit. |
| 678 |
|
| 679 |
* Added harp_product_bin_spatial() and harp_variable_copy_attributes() |
| 680 |
functions. |
| 681 |
|
| 682 |
* Added optional parameter to regrid() operation to specify the axis boundaries. |
| 683 |
|
| 684 |
* Python harp.import_product() now ignores empty products when ingesting a list |
| 685 |
of products. |
| 686 |
|
| 687 |
* Added derivations: |
| 688 |
- column vmr/mmr from column density ratios |
| 689 |
- total column mass density from surface pressure and pressure profile |
| 690 |
- mass mixing ratio from mass density |
| 691 |
- between dry air and total air mass (column) density |
| 692 |
- between dry air, total air, and H2O number densities |
| 693 |
|
| 694 |
* Fixed derivation of altitude from gph/latitude. |
| 695 |
|
| 696 |
* Updates to S5P L2 and MIPAS L2 product ingestions. |
| 697 |
|
| 698 |
1.2 2018-04-09 |
| 699 |
~~~~~~~~~~~~~~ |
| 700 |
|
| 701 |
* Allow data compression for HDF5 output: |
| 702 |
- Added harp_set_option_hdf5_compression() and |
| 703 |
harp_get_option_hdf5_compression() functions to C library. |
| 704 |
- Added --hdf5-compression option to harpconvert and harpmerge. |
| 705 |
- Added hdf5_compression parameter to harp.export_product() Python function. |
| 706 |
|
| 707 |
* Allow regrid() operation to interpolate in the time dimension. |
| 708 |
|
| 709 |
* Improved binning of angle values (average of 1 and 359 [deg] is now 0). |
| 710 |
|
| 711 |
* Added bin() operation variant without parameters. |
| 712 |
|
| 713 |
* The bin() operation using a collocation result file now retains the |
| 714 |
collocation_index variable (with the first value for each bin). |
| 715 |
|
| 716 |
* Changed smooth() operation such that each vertical level where the source |
| 717 |
grid point was NaN the target grid point also remains NaN. |
| 718 |
|
| 719 |
* Added -ap option to harpmerge. |
| 720 |
|
| 721 |
* Added --dataset option to harpdump and changed output format of |
| 722 |
harp_dataset_print() and harp_product_metadata_print() to csv format. |
| 723 |
|
| 724 |
* Added 'operations' parameter to Python harp.export_product() function. |
| 725 |
|
| 726 |
* Importing/exporting a product with operations in Python now automatically |
| 727 |
updates the history attribute. |
| 728 |
|
| 729 |
* Fixed issue where harpmerge did not introduce a time dimension for all |
| 730 |
variables. |
| 731 |
|
| 732 |
* Fixed issue where datetime_start and datetime_stop global attributes were not |
| 733 |
created when exporting in HDF4 or HDF5 format. |
| 734 |
|
| 735 |
* String attributes in HDF5 files can now be empty strings (using H5S_NULL). |
| 736 |
|
| 737 |
* Allow HARP to import HDF5 files that are generated by the netCDF4 library |
| 738 |
(allow numeric attributes to be 1-element arrays instead of scalars). |
| 739 |
|
| 740 |
* harpcheck now returns with exit code 1 if any of the imports fail. |
| 741 |
|
| 742 |
* harpcollocate --update now only allows one dataset as argument. |
| 743 |
|
| 744 |
* Fixed performance issue with spatial filtering on systems with slow math lib. |
| 745 |
|
| 746 |
* Fixed mapping description of GOME2 L1 product ingestion. |
| 747 |
|
| 748 |
* Fixed wrong results of area overlap calculation. |
| 749 |
|
| 750 |
* Replaced '_' by '-' in long versions of argument names of harpcollocate. |
| 751 |
|
| 752 |
* Added support for dfs (degrees of freedom for signal) profiles. |
| 753 |
|
| 754 |
* Updates to S5P L2 product ingestions. |
| 755 |
|
| 756 |
1.1 2017-11-06 |
| 757 |
~~~~~~~~~~~~~~ |
| 758 |
|
| 759 |
* Parameters that required a file or a directory as dataset reference can now |
| 760 |
also take a file with a .pth extension as input. This file should contain a |
| 761 |
line-separated list of paths to files and/or directories of the dataset. |
| 762 |
|
| 763 |
* Specifying a directory for a dataset will now recursively add all products |
| 764 |
(including sub-directories and contents of .pth files). |
| 765 |
|
| 766 |
* The variable that is binned on with a 'bin(variable)' operation is now |
| 767 |
always kept in the resulting product. |
| 768 |
|
| 769 |
* The bin() operation can now handle 'count' variables (summing it if present |
| 770 |
and creating it otherwise). |
| 771 |
|
| 772 |
* Fixed issue where collocate filter operation on a product with a single |
| 773 |
sample could end in an infinite loop. |
| 774 |
|
| 775 |
* Fixed issue in harpcollocate when providing a -nx option without a -ny. |
| 776 |
|
| 777 |
* Fixed issue with wrap() operation for values < specified minimum. |
| 778 |
|
| 779 |
* Fixed issue with collocate_left/collocate_right operations not including |
| 780 |
last sample when applying it on products in an external format. |
| 781 |
|
| 782 |
* harpcollocate now ignores empty products instead of raising errors for them. |
| 783 |
|
| 784 |
* Added -oa/-ob and -aa/-ab options to harpcollocate matchup. |
| 785 |
|
| 786 |
* Added solar_declination_angle and solar_hour_angle variables and derivations. |
| 787 |
|
| 788 |
* Added orbit_index variable and added this to various product ingestions. |
| 789 |
|
| 790 |
* Fixed derivations of solar azimuth/zenith angles from datetime and lat/lon. |
| 791 |
|
| 792 |
* Added derivations for averaging kernel conversions. |
| 793 |
|
| 794 |
* Added derivations for tropopause altitude/pressure. |
| 795 |
|
| 796 |
* Added NH3 (amonia) as supported species. |
| 797 |
|
| 798 |
* Updates to ECMWF-GRIB, GEOMS-TE-UVVIS-DOAS-OFFAXIS, GEOMS-TE-FTIR, and |
| 799 |
S5P L2 product ingestions (and several others for the new orbit_index). |
| 800 |
|
| 801 |
1.0 2017-09-25 |
| 802 |
~~~~~~~~~~~~~~ |
| 803 |
|
| 804 |
* The import and ingest functions have been combined into a single import |
| 805 |
function that can both read HARP compliant products as well as ingest from |
| 806 |
the supported external formats. This means: |
| 807 |
- harp_import() and harp_ingest() are combined into a single harp_import() |
| 808 |
- harp_ingest_test() has been renamed to harp_import_test() |
| 809 |
- harpfilter has been removed (use harpconvert instead) |
| 810 |
- '--test' option of harpconvert has been removed (use harpcheck instead) |
| 811 |
|
| 812 |
* harpcollocate has been completely rewritten. |
| 813 |
- any 1D time-dependent variable can now be a collocation criterium |
| 814 |
- nearest neighbour filtering with weighted average is no longer supported |
| 815 |
- collocation result files are not backward compatible due to a change in |
| 816 |
the CSV header |
| 817 |
- arguments to harp_collocation_result_new() and |
| 818 |
harp_collocation_result_add_pair() have changed |
| 819 |
|
| 820 |
* Added support for enumeration variables. Enumeration variables have a int8, |
| 821 |
int16, or int32 type and provide an index into a fixed list of enum names. |
| 822 |
Enumeration variables can be treated as string values in operations. |
| 823 |
|
| 824 |
* Added MATLAB and IDL interfaces for HARP. |
| 825 |
|
| 826 |
* Added -o option to harpdump and harpmerge. |
| 827 |
|
| 828 |
* Added -a and -o options for 'harpdump --list-derivations'. |
| 829 |
|
| 830 |
* '--list-derivations' option of harpconvert has been removed (use harpdump |
| 831 |
instead). |
| 832 |
|
| 833 |
* Added -l option to harpmerge. |
| 834 |
|
| 835 |
* Renamed area/point operations: |
| 836 |
- area_mask_covers_area -> area_covers_area |
| 837 |
- area_mask_covers_point -> point_in_area |
| 838 |
- area_mask_intersects_area -> area_intersects_area |
| 839 |
- point_in_area -> area_covers_point |
| 840 |
The first part of the operation name now reflects the item in the |
| 841 |
product and the second part reflects the parameter that is provided. |
| 842 |
|
| 843 |
* Improved performance of point/area vs. area overlap operations by first |
| 844 |
comparing against the bounding box of a polygon. |
| 845 |
|
| 846 |
* Added sort() and area_inside_area() operations. |
| 847 |
|
| 848 |
* The derive() operation can now be used without dimension specification |
| 849 |
to perform a unit and/or data type conversion on an existing variable. |
| 850 |
|
| 851 |
* The derive_smoothed_column(), regrid() and smooth() operations can now |
| 852 |
use a single merged collocated product (instead of a combination of a csv |
| 853 |
collocation result file and a dataset directory of collocated products). |
| 854 |
|
| 855 |
* An area parameter for operations can now either be a reference to an |
| 856 |
area mask file or a single polygon provided as lat/lon array parameters. |
| 857 |
|
| 858 |
* Added derivation to derive the area (size) of latitude/longitude bounds. |
| 859 |
|
| 860 |
* Added derivation to derive degree of freedom for signal (DFS) from 2D |
| 861 |
averaging kernels (AVK). |
| 862 |
|
| 863 |
* Derivations of bound variables will now clamp to the edge midpoints when |
| 864 |
the 'regrid_out_of_bounds' option is set to 'edge'. |
| 865 |
|
| 866 |
* The 'data_type' parameter for harp_product_get_derived_variable() and |
| 867 |
harp_product_add_derived_variable() is now a pointer to allow it to be |
| 868 |
optional (i.e. pass NULL to use the default data type). |
| 869 |
|
| 870 |
* Added harp_geometry_get_area(), |
| 871 |
harp_product_get_smoothed_column_using_collocated_product(), |
| 872 |
harp_product_smooth_vertical_with_collocated_product(), |
| 873 |
harp_product_sort(), harp_product_regrid_with_collocated_product(), |
| 874 |
harp_set_udunits2_xml_path(), harp_set_udunits2_xml_path_conditional(), |
| 875 |
and harp_variable_set_enumeration_values() functions. |
| 876 |
|
| 877 |
* harp_import_product_metadata() and harp_dataset_import() now have an |
| 878 |
'options' parameter. |
| 879 |
|
| 880 |
* Changed 'percentage' parameter of harp_geometry_has_area_overlap() to |
| 881 |
'fraction'. |
| 882 |
|
| 883 |
* harp_product_set_source_product() now has a product path as parameter |
| 884 |
and stores the base name of this as source_product value. |
| 885 |
|
| 886 |
* Added HARP_UNIT_AREA default unit. |
| 887 |
|
| 888 |
* The C interface for collocation results was modified to support a |
| 889 |
dynamic set of collocation criteria differences (based on variable names |
| 890 |
for these differences). |
| 891 |
|
| 892 |
* The Python interface now sets the udunits2 xml path dynamically, |
| 893 |
relative to the location of the HARP C library. |
| 894 |
|
| 895 |
* wrap() operation now properly updates valid_min/valid_max properties. |
| 896 |
|
| 897 |
* codadef files are now installed in <prefix>/share/coda/definitions. |
| 898 |
This allows a present CODA installation to use the same codadef files. |
| 899 |
|
| 900 |
* Windows installation now installs data and documentation files in the |
| 901 |
subdirectory <prefix>/share/harp. |
| 902 |
|
| 903 |
* Fix issue in deriving total column from partial column profile if partial |
| 904 |
column profile did not have {time,vertical} dimensions. |
| 905 |
|
| 906 |
* Fix invalid results from binning operation/function. |
| 907 |
|
| 908 |
* Improve speed of --list-derivations on a product. |
| 909 |
|
| 910 |
* Split data format documentation into separate sections under a new |
| 911 |
heading 'conventions'. |
| 912 |
|
| 913 |
* Added product ingestions for Cloudnet L2a classification products. |
| 914 |
|
| 915 |
* Updates to ECMWF-GRIB, GEOMS-TE-UVVIS-DOAS-OFFAXIS, MIPAS, QA4ECV, and |
| 916 |
S5P product ingestions. |
| 917 |
|
| 918 |
0.6 2017-04-23 |
| 919 |
~~~~~~~~~~~~~~ |
| 920 |
|
| 921 |
* Added ingestion support for ACE-FTS, CALIPSO, Cloud CCI, and NPP-Suomi. |
| 922 |
|
| 923 |
* Added bin() and set() operations. |
| 924 |
|
| 925 |
* Added optional data_type parameter to derive() operation. |
| 926 |
|
| 927 |
* Major refactoring of internal implementation of operations (greatly |
| 928 |
reducing and simplifying the code). |
| 929 |
|
| 930 |
* Fix application of apriori for smoothed column derivation (for vertical |
| 931 |
range where partial columns are NaN, don't subtract apriori*AVK, but just |
| 932 |
set to apriori). |
| 933 |
|
| 934 |
* Fix memory issue in partial column from vmr dry air derivation. |
| 935 |
|
| 936 |
* Remove double definition of radiance conversions. |
| 937 |
|
| 938 |
* Added global HARP option 'regrid_out_of_bounds'. |
| 939 |
|
| 940 |
* Added harp_product_bin() function. |
| 941 |
|
| 942 |
* Add data_type parameter to harp_product_add_derived_variable() and |
| 943 |
harp_product_get_derived_variable() functions. |
| 944 |
|
| 945 |
* Added data_type parameter to harp_product_get_derived_variable() and |
| 946 |
harp_product_add_derived_variable() functions. |
| 947 |
|
| 948 |
* Allow 'Conventions' global attribute to contain more than just a reference |
| 949 |
to HARP. |
| 950 |
|
| 951 |
* Renamed O3MSAF codadef to ACSAF. When updating HARP, you should remove |
| 952 |
any existing O3SMAF-*.codadef files from your installation directory. |
| 953 |
Otherwise you will get 'detection rule for ... is shadowed by ... ' errors. |
| 954 |
|
| 955 |
* Fixed issue with TES TL2CH4L ingestion. |
| 956 |
|
| 957 |
* Updates to Aeolus, GEOMS, S5P, QA4ECV, and Ozone CCI product ingestions. |
| 958 |
|
| 959 |
0.5 2017-03-26 |
| 960 |
~~~~~~~~~~~~~~ |
| 961 |
|
| 962 |
* Changed Open Source licence of HARP from GPLv2 to BSD (3-Clause). |
| 963 |
|
| 964 |
* Changed all '-' to '_' in operation function names to make them identifiers. |
| 965 |
|
| 966 |
* Added harpmerge command line tool. |
| 967 |
|
| 968 |
* Added regrid(), rename(), smooth(), derive_smoothed_column(), |
| 969 |
point_in_area(), and wrap() operations. |
| 970 |
|
| 971 |
* Removed the harpprofile command line tool. Its functionality is replaced by |
| 972 |
the regrid() and smooth() operations. |
| 973 |
|
| 974 |
* Renamed harp_product_regrid_vertical_with_collocated_dataset() to |
| 975 |
harp_product_regrid_with_collocated_dataset() and |
| 976 |
harp_product_smooth_vertical() to |
| 977 |
harp_product_smooth_vertical_with_collocated_dataset() |
| 978 |
|
| 979 |
* Added vertical_unit parameter to harp_product_smooth_vertical() and |
| 980 |
harp_product_regrid_with_collocated_dataset() functions. |
| 981 |
|
| 982 |
* Replaced harp_product_regrid_vertical_with_axis_variable() with |
| 983 |
harp_product_regrid_with_axis_variable(). It can now be used for other |
| 984 |
dimensions too (except the time dimension). |
| 985 |
|
| 986 |
* Added harp_collocation_result_filter_for_collocation_indices(), |
| 987 |
harp_product_get_smoothed_column(), |
| 988 |
harp_product_get_smoothed_column_using_collocated_dataset(), |
| 989 |
harp_product_remove_variable_by_name(), harp_variable_append(), |
| 990 |
harp_product_append(), and harp_variable_smooth_vertical() functions. |
| 991 |
|
| 992 |
* Renamed harp_product_get_variable_id_by_name() to |
| 993 |
harp_product_get_variable_index_by_name(). |
| 994 |
|
| 995 |
* Python harp.import_product() and harp.ingest_product() functions now accept |
| 996 |
a list of filenames or a globbing pattern as parameter (which will return |
| 997 |
the result of harp.concatenate() on the imported/ingested products). |
| 998 |
|
| 999 |
* Automatically set CODA definition path for HARP Python interface |
| 1000 |
(path can be overruled with CODA_DEFINITION environment variable). |
| 1001 |
|
| 1002 |
* The parsing of the operations string has been rewritten using flex/yacc. |
| 1003 |
This has fixed several issues related to the parsing and execution of |
| 1004 |
operations. |
| 1005 |
|
| 1006 |
* Fixed issue with flatten() operation providing wrong results. |
| 1007 |
|
| 1008 |
* Fixed issue where vertical regridding to a common grid provided wrong |
| 1009 |
results. |
| 1010 |
|
| 1011 |
* Fixed issue where interval interpolation for a pressure grid was not |
| 1012 |
using a log-linear interpolation. |
| 1013 |
|
| 1014 |
* Aligned order of latitude/longitude arguments in all functions to have |
| 1015 |
latitude first and longitude second. |
| 1016 |
|
| 1017 |
* Added unit 'ppv' to the included udunits2-common.xml file. |
| 1018 |
|
| 1019 |
* Added variables for surface temperature, total water vapour content, and |
| 1020 |
total cloud cover to ECMWF GRIB ingestion. |
| 1021 |
|
| 1022 |
* The default internal units have been changes to SI (except for molar mass). |
| 1023 |
All algorithms have been adapted accordingly. |
| 1024 |
|
| 1025 |
* Added variable conversions for: apriori profiles, column density from |
| 1026 |
mass density (including pm1/pm2.5/pm10). |
| 1027 |
|
| 1028 |
* Added support for AEOLUS L1b/L2a/L2b products. |
| 1029 |
|
| 1030 |
* Added support for Aerosol and GHG CCI L2/L3 products. |
| 1031 |
|
| 1032 |
* Added support for GEOMS FTIR C2H2 ingestion. |
| 1033 |
|
| 1034 |
* Added support for OMI OMCHOCHO ingestion. |
| 1035 |
|
| 1036 |
* Added support for additional TES L2 products. |
| 1037 |
|
| 1038 |
* Added support for QA4ECV NO2 and HCHO products. |
| 1039 |
|
| 1040 |
* Added support for GOME, GOME-2, GOMOS, MIPAS, IASI, SCIAMACHY L1 products. |
| 1041 |
|
| 1042 |
* Added support for ODIN OSIRIS and SMR L2 products. |
| 1043 |
|
| 1044 |
* Changed --list-conversions command line option to --list-derivations. |
| 1045 |
And added --list-derivations option to harpdump. |
| 1046 |
|
| 1047 |
* Use NC_64BIT_OFFSET when storing data to netcdf larger than 1GB. |
| 1048 |
|
| 1049 |
0.4 2016-10-16 |
| 1050 |
~~~~~~~~~~~~~~ |
| 1051 |
|
| 1052 |
* Fixed wrong scaling factor in conversions related to the ideal gas law |
| 1053 |
(10^2 vs. 10^-2) and barometric formula (10^-9 vs. 10^-6). |
| 1054 |
|
| 1055 |
* Fixed error in algorithm to derive gravity from latitude and height. |
| 1056 |
|
| 1057 |
* Fixed invalid sorting of collocation result file (which resulted in wrong |
| 1058 |
results when performing a collocation resampling). |
| 1059 |
|
| 1060 |
* Fixed wrong independent dimension length for altitude/pressure bounds for |
| 1061 |
SCIAMACHY Level2 ingestion. |
| 1062 |
|
| 1063 |
* Added apriori number density variables to SCIAMACHY Level2 ingestion. |
| 1064 |
|
| 1065 |
* Added variable conversion from latitude/longitude per vertical profile point |
| 1066 |
to single (mid-point) latitude/longitude for profile. |
| 1067 |
|
| 1068 |
* Variable conversions for altitude and pressure bounds now also work when |
| 1069 |
they are dependend on latitude and longitude dimensions. |
| 1070 |
|
| 1071 |
* Fixed crash in derived variable conversion for volume mixing ratio from |
| 1072 |
mass mixing ratio (for total air). |
| 1073 |
|
| 1074 |
* Changed dimension argument of 'flatten()' operation from string to |
| 1075 |
identifier. |
| 1076 |
|
| 1077 |
* Fixed issue where -inf/+inf valid_min/valid_max attributes could not be |
| 1078 |
exported as float values in the Python harp.export_product() interface. |
| 1079 |
|
| 1080 |
* When deriving latitude/longitude bounds from mid points, latitude is now |
| 1081 |
clamped to [-90,90] and longitude is wrapped to [-180,180]. |
| 1082 |
|
| 1083 |
* Fixed volume mixing ratio unit for GEOMS-TE-UVVIS-DOAS-OFFAXIS ingestion. |
| 1084 |
|
| 1085 |
0.3 2016-09-30 |
| 1086 |
~~~~~~~~~~~~~~ |
| 1087 |
|
| 1088 |
* The changes between 0.2 and 0.3 are extensive. The list below provides the |
| 1089 |
most important aspects, but there are many more smaller changes that are not |
| 1090 |
listed. |
| 1091 |
|
| 1092 |
* HARP 'actions' are now called 'operations' and are now executed in a strict |
| 1093 |
sequence. This means that you can now do things like: |
| 1094 |
'derive(var); var < 10; exclude(var)' |
| 1095 |
|
| 1096 |
* Added harpprofile command line tool that allows regridding and smoothing |
| 1097 |
of vertical gridded data. |
| 1098 |
Also added harp_product_regrid_vertical_with_axis_variable(), |
| 1099 |
harp_product_regrid_vertical_with_collocated_dataset(), and |
| 1100 |
harp_product_smooth_vertical() to C interface. |
| 1101 |
|
| 1102 |
* Added algorithm descriptions for the derived variable conversions to the |
| 1103 |
HARP documentation. |
| 1104 |
|
| 1105 |
* Introduced a more strict definition of variable names for the HARP data |
| 1106 |
format. As part of this the list of supported species and associated naming |
| 1107 |
has also been revised. |
| 1108 |
|
| 1109 |
* Introduced strict separation between dry air and total/wet air. The default |
| 1110 |
air density in HARP is now the total/wet air. Anything related to dry air |
| 1111 |
should now be annotated as such in the variable name. As a result, 'dry_air' |
| 1112 |
is now introduced as a species, and for mixing ratios the '_wet' postfix is |
| 1113 |
removed and the '_dry_air' postfix is introduced. |
| 1114 |
Derived variable conversions have been added for dry/wet conversion and for |
| 1115 |
retrieving the needed 'molar mass' of total air. |
| 1116 |
|
| 1117 |
* Added functionality to deal with HARP Datasets and Product Metadata to the |
| 1118 |
public C library interface. The collocation result interface in the C |
| 1119 |
libraryinterface has also been reworked to now work with HARP Datasets. |
| 1120 |
|
| 1121 |
* HARP_ERROR_ACTION and HARP_ERROR_ACTION_SYNTAX have been renamed to |
| 1122 |
HARP_ERROR_OPERATION and HARP_ERROR_OPERATION_SYNTAX. |
| 1123 |
harp_product_execute_actions() has been renamed to |
| 1124 |
harp_product_execute_operations(). |
| 1125 |
|
| 1126 |
* Added operation to flatten data in a certain dimension and added |
| 1127 |
harp_product_flatten_dimension(). |
| 1128 |
|
| 1129 |
* Automatic detection of a product type when ingesting products now happens |
| 1130 |
using CODA (using included codadef files with detection rules) for all |
| 1131 |
products. |
| 1132 |
|
| 1133 |
* harp_import_global_attributes() has been removed (its functionality is |
| 1134 |
now captured by the new Product Metadata interface). |
| 1135 |
|
| 1136 |
* Added public functions to allow raising/capturing of warnings in HARP: |
| 1137 |
harp_report_warning(), harp_get_warning_handler(), harp_set_warning_handler() |
| 1138 |
|
| 1139 |
* Added public functions to print product/variable structure and content: |
| 1140 |
harp_product_print(), harp_variable_print(), harp_variable_print_data() |
| 1141 |
|
| 1142 |
* Fixed issue in the HARP Python interface where using the value 0 for |
| 1143 |
valid_min or valid_max did not work. |
| 1144 |
|
| 1145 |
* Added concatenate() function to HARP Python interface. |
| 1146 |
|
| 1147 |
* The 'include()' action/operation has been renamed to 'keep()'. |
| 1148 |
|
| 1149 |
* The derived variable conversions are now properly split according to the |
| 1150 |
dimension types of a variable. This also solves a problem where certain |
| 1151 |
valid conversions where not included in a '--list-conversions' print. |
| 1152 |
|
| 1153 |
* The derived variable conversions that produce uncertainty and covariance |
| 1154 |
variables filled with NaNs from scratch have been removed. |
| 1155 |
|
| 1156 |
* Added derived variable conversions for gridded data (i.e. variables that |
| 1157 |
depend on the latitude and longitude dimensions). |
| 1158 |
|
| 1159 |
* Removed variables flag_am_pm and flag_day_twilight_night and the conversions |
| 1160 |
for flag_am_pm, flag_day_twilight_night, and flag_daytime. These conditions |
| 1161 |
can all be determined by directly comparing against the solar angles. |
| 1162 |
|
| 1163 |
* Added support for ingestion of GOMOS L2, SCIAMACHY L2, ECMF GRIB and TEMIS |
| 1164 |
assimilated ozone data. |
| 1165 |
|
| 1166 |
* Added 'species' ingestion option to the MIPAS L2 ingestion. |
| 1167 |
|
| 1168 |
* Added support for MLS CH3Cl, CH3CN, and CH3OH products. |
| 1169 |
|
| 1170 |
* Fixed issue in GEOMS ingestion where long variable names were truncated. |
| 1171 |
|
| 1172 |
* Introduced convention that dimensionless quantities should have an empty |
| 1173 |
'units' attribute string value. Only non-quantities should omit the 'units' |
| 1174 |
attribute. |
| 1175 |
|
| 1176 |
* Resolved incorrect correction for leap seconds in product ingestions. |
| 1177 |
|
| 1178 |
* Covariance variables have been renamed from '*_cov' to '*_covariance'. |
| 1179 |
|
| 1180 |
* The split of random/systematic for covariances has been removed. |
| 1181 |
For ingested products that had such covariances, the random covariances are |
| 1182 |
now retrieved as plain covariances and from systematic covariances only the |
| 1183 |
trace is retrieved and provided as a systematic uncertainty. |
| 1184 |
|
| 1185 |
* Added 'geopotential' as a supported quantity with conversions from/to |
| 1186 |
'geopotential height'. Added HARP_UNIT_GEOPOTENTIAL. |
| 1187 |
|
| 1188 |
* Renamed cloud optical thickness to cloud optical depth. |
| 1189 |
|
| 1190 |
* Added '-a/--operations' option to harpdump. |
| 1191 |
|
| 1192 |
* Support for HDF4 and HDF5 (for import/export) is now optional when building |
| 1193 |
HARP, but still enabled by default. |
| 1194 |
|
| 1195 |
0.2 2016-06-30 |
| 1196 |
~~~~~~~~~~~~~~ |
| 1197 |
|
| 1198 |
* Added Python interface. |
| 1199 |
|
| 1200 |
* CODA is no longer included as part of HARP, but is treated as a separate |
| 1201 |
package, just like HDF4 and HDF5. |
| 1202 |
|
| 1203 |
* The CMake build system can now be used to build HARP on Windows and create |
| 1204 |
a binary installer (using WIX). |
| 1205 |
|
| 1206 |
* Revised and renumbered error codes in C interface. |
| 1207 |
Renamed HARP_ERROR_SCRIPT and HARP_ERROR_SCRIPT_SYNTAX error codes to |
| 1208 |
HARP_ERROR_ACTION and HARP_ERROR_ACTION_SYNTAX. Changed HARP_ERROR_PRODUCT |
| 1209 |
to either HARP_ERROR_INGESTION or HARP_ERROR_IMPORT depending on context. |
| 1210 |
Added HARP_ERROR_EXPORT, HARP_ERROR_INVALID_PRODUCT and |
| 1211 |
HARP_ERROR_INVALID_VARIABLE. |
| 1212 |
|
| 1213 |
* The HARP command line tools now link against the dynamic HARP library |
| 1214 |
instead of the static version. |
| 1215 |
|
| 1216 |
* Updated embedded expat library to version 2.1.1. |
| 1217 |
|
| 1218 |
* Updated embedded udunits2 library to version 2.2.20. |
| 1219 |
Also removed the HARP specific udunits XML configuration file since all |
| 1220 |
HARP units are now included in udunits2. |
| 1221 |
|
| 1222 |
* Renamed all 'stdev' variables to 'uncertainty'. |
| 1223 |
|
| 1224 |
* Fixed parsing of dimension names; string dimension could be interpreted as |
| 1225 |
independent dimension. |
| 1226 |
|
| 1227 |
* Improved the efficiency of the conversion from volume mixing ratio to number |
| 1228 |
density and vice versa. |
| 1229 |
|
| 1230 |
* Update of S5P L2 ingestion module to the lastest version of the product |
| 1231 |
format specification (S5P-KNMI-L2-0009-SD and relevant DLR PUMs). |
| 1232 |
|
| 1233 |
* Update of S5P L1B ingestion module to the lastest version of the product |
| 1234 |
format specification (S5P-KNMI-L01B-0012-SD). |
| 1235 |
|
| 1236 |
* Added ingestion modules for the following product types: |
| 1237 |
- Ozone CCI L2 limb profile |
| 1238 |
- Ozone CCI L3 limb profile |
| 1239 |
- Ozone CCI L2 nadir profile |
| 1240 |
- Ozone CCI L3 nadir profile |
| 1241 |
- Ozone CCI L4 nadir profile |
| 1242 |
- Ozone CCI L2 total column |
| 1243 |
- Ozone CCI L3 total column |
| 1244 |
|
| 1245 |
* Added public functions harp_get_fill_value_for_type(), |
| 1246 |
harp_get_valid_min_for_type(), harp_get_valid_max_for_type(), |
| 1247 |
harp_is_fill_value_for_type(), harp_is_valid_min_for_type(), |
| 1248 |
harp_is_valid_max_for_type(), harp_ingest_test(), |
| 1249 |
harp_doc_list_conversions(), harp_doc_export_ingestion_definitions(), |
| 1250 |
harp_geometry_get_point_distance(), harp_geometry_has_point_in_area(), |
| 1251 |
harp_geometry_has_area_overlap(), harp_basename(), |
| 1252 |
harp_product_execute_actions(), harp_variable_set_description(), |
| 1253 |
harp_variable_set_unit(), harp_variable_set_string_data_element(), |
| 1254 |
harp_variable_verify(), harp_product_set_source_product(), |
| 1255 |
harp_product_set_history(), harp_product_verify() |
| 1256 |
|
| 1257 |
* Added public functions for dealing with HARP collocations. |
| 1258 |
|
| 1259 |
* Moved defines for default harp units to harp.h. |
| 1260 |
|
| 1261 |
* Added operators =& and !& to the product actions to test integer variables |
| 1262 |
against a bit mask, where a =& mask is equivalent to (a & mask) != 0 and |
| 1263 |
a !& mask is equivalent to (a & mask) == 0. |
| 1264 |
|
| 1265 |
* Added functions to verify the internal consistency as well as compliance to |
| 1266 |
HARP conventions of harp_product and harp_variable instances. Any instance |
| 1267 |
of harp_product returned by harp_import() or harp_ingest() has been verified |
| 1268 |
using these functions. |
| 1269 |
|
| 1270 |
* Added derived variable conversion from 'latitude {latitude}' (mid-points) to |
| 1271 |
'latitude_bounds {latitude, independent}' (bounds); similar for longitude. |
| 1272 |
|
| 1273 |
* Updated HDF5 data format such that HDF5 products written by HARP are |
| 1274 |
compatible with netCDF-4. HDF5 dimension scales are used to represent shared |
| 1275 |
dimensions. The "dims" dataset attribute has been deprecated. |
| 1276 |
|
| 1277 |
* Changed netCDF-3/HDF4/HDF5 backends to be more explicit about implicitly |
| 1278 |
created dimensions (which are used to store scalars and strings for formats |
| 1279 |
that do not support this natively). |
| 1280 |
|
| 1281 |
* Fixed handling of scalars and empty strings in netCDF-3/HDF4/HDF5 backends. |
| 1282 |
|
| 1283 |
* Fixed unit attribute name in HARP products ('unit' -> 'units'). |
| 1284 |
|
| 1285 |
* Changed all constants to be in line with CODATA 2014 as published by NIST. |
| 1286 |
|
| 1287 |
0.1 2015-11-09 |
| 1288 |
~~~~~~~~~~~~~~ |
| 1289 |
|
| 1290 |
* First beta release (using CODA 2.15). |