| 1 |
Source: kerchunk |
| 2 |
Section: python |
| 3 |
Maintainer: Debian GIS Project <[email protected]> |
| 4 |
Uploaders: Antonio Valentino <[email protected]> |
| 5 |
Build-Depends: bitshuffle, |
| 6 |
debhelper-compat (= 13), |
| 7 |
dh-sequence-python3, |
| 8 |
dh-sequence-sphinxdoc <!nodoc>, |
| 9 |
hdf5-filter-plugin, |
| 10 |
pybuild-plugin-pyproject, |
| 11 |
python3-aiohttp <!nocheck>, |
| 12 |
python3-all, |
| 13 |
python3-astropy <!nodoc>, |
| 14 |
python3-dask <!nocheck>, |
| 15 |
python3-cfgrib, |
| 16 |
python3-cftime, |
| 17 |
python3-eccodes <!nodoc>, |
| 18 |
python3-fsspec, |
| 19 |
python3-h5netcdf <!nocheck>, |
| 20 |
python3-h5py, |
| 21 |
python3-hdf5plugin, |
| 22 |
python3-netcdf4 <!nocheck>, |
| 23 |
python3-numcodecs, |
| 24 |
python3-numpy, |
| 25 |
python3-numpydoc <!nodoc>, |
| 26 |
python3-pytest <!nocheck>, |
| 27 |
python3-s3fs <!nocheck>, |
| 28 |
python3-scipy, |
| 29 |
python3-setuptools, |
| 30 |
python3-setuptools-scm, |
| 31 |
python3-sphinx <!nodoc>, |
| 32 |
python3-sphinx-rtd-theme <!nodoc>, |
| 33 |
python3-tifffile <!nodoc>, |
| 34 |
python3-ujson, |
| 35 |
python3-xarray, |
| 36 |
python3-zarr |
| 37 |
Standards-Version: 4.7.4 |
| 38 |
Testsuite: autopkgtest-pkg-pybuild |
| 39 |
Homepage: https://github.com/fsspec/kerchunk |
| 40 |
Vcs-Browser: https://salsa.debian.org/debian-gis-team/kerchunk |
| 41 |
Vcs-Git: https://salsa.debian.org/debian-gis-team/kerchunk.git |
| 42 |
Description: Cloud-friendly access to archival data |
| 43 |
Kerchunk is a library that provides a unified way to represent a |
| 44 |
variety of chunked, compressed data formats (e.g. NetCDF, HDF5, GRIB), |
| 45 |
allowing efficient access to the data from traditional file systems or |
| 46 |
cloud object storage. It also provides a flexible way to create |
| 47 |
virtual datasets from multiple files. It does this by extracting the |
| 48 |
byte ranges, compression information and other information about the |
| 49 |
data and storing this metadata in a new, separate object. |
| 50 |
This means that you can create a virtual aggregate dataset over |
| 51 |
potentially many source files, for efficient, parallel and |
| 52 |
cloud-friendly *in-situ* access without having to copy or translate |
| 53 |
the originals. It is a gateway to in-the-cloud massive data processing |
| 54 |
while the data providers still insist on using legacy formats for |
| 55 |
archival storage. |
| 56 |
. |
| 57 |
Features: |
| 58 |
. |
| 59 |
* completely serverless architecture |
| 60 |
* metadata consolidation, so you can understand a many-file dataset |
| 61 |
(metadata plus physical storage) in a single read |
| 62 |
* read from all of the storage backends supported by fsspec, |
| 63 |
including object storage (s3, gcs, abfs, alibaba), http, cloud user |
| 64 |
storage (dropbox, gdrive) and network protocols (ftp, ssh, hdfs, |
| 65 |
smb...) |
| 66 |
* loading of various file types (currently netcdf4/HDF, grib2, tiff, |
| 67 |
fits, zarr), potentially heterogeneous within a single dataset, |
| 68 |
without a need to go via the specific driver (e.g., no need for |
| 69 |
h5py) |
| 70 |
* asynchronous concurrent fetch of many data chunks in one go, |
| 71 |
amortizing the cost of latency |
| 72 |
* parallel access with a library like zarr without any locks |
| 73 |
* logical datasets viewing many (>~millions) data files, and direct |
| 74 |
access/subselection to them via coordinate indexing across an |
| 75 |
arbitrary number of dimensions |
| 76 |
|
| 77 |
Package: python3-kerchunk |
| 78 |
Architecture: all |
| 79 |
Depends: ${python3:Depends}, |
| 80 |
${misc:Depends} |
| 81 |
Recommends: hdf5-filter-plugin, |
| 82 |
python3-cfgrib, |
| 83 |
python3-cftime, |
| 84 |
python3-h5py, |
| 85 |
python3-hdf5plugin, |
| 86 |
python3-scipy, |
| 87 |
python3-xarray |
| 88 |
Suggests: bitshuffle, |
| 89 |
python3-aiohttp, |
| 90 |
python3-dask, |
| 91 |
python3-netcdf4, |
| 92 |
python3-s3fs |
| 93 |
Description: ${source:Synopsis} |
| 94 |
${source:Extended-Description} |
| 95 |
|
| 96 |
Package: python-kerchunk-doc |
| 97 |
Section: doc |
| 98 |
Architecture: all |
| 99 |
Multi-Arch: foreign |
| 100 |
Depends: ${sphinxdoc:Depends}, |
| 101 |
${misc:Depends} |
| 102 |
Suggests: www-browser |
| 103 |
Description: ${source:Synopsis} (documentation) |
| 104 |
${source:Extended-Description} |
| 105 |
. |
| 106 |
This package provides the HTML documentation for kerchunk. |