Debian Codegraph @

debian/control

raw
1 Source: kerchunk
2 Section: python
3 Maintainer: Debian GIS Project <[email protected]>
4 Uploaders: Antonio Valentino <[email protected]>
5 Build-Depends: bitshuffle,
6 debhelper-compat (= 13),
7 dh-sequence-python3,
8 dh-sequence-sphinxdoc <!nodoc>,
9 hdf5-filter-plugin,
10 pybuild-plugin-pyproject,
11 python3-aiohttp <!nocheck>,
12 python3-all,
13 python3-astropy <!nodoc>,
14 python3-dask <!nocheck>,
15 python3-cfgrib,
16 python3-cftime,
17 python3-eccodes <!nodoc>,
18 python3-fsspec,
19 python3-h5netcdf <!nocheck>,
20 python3-h5py,
21 python3-hdf5plugin,
22 python3-netcdf4 <!nocheck>,
23 python3-numcodecs,
24 python3-numpy,
25 python3-numpydoc <!nodoc>,
26 python3-pytest <!nocheck>,
27 python3-s3fs <!nocheck>,
28 python3-scipy,
29 python3-setuptools,
30 python3-setuptools-scm,
31 python3-sphinx <!nodoc>,
32 python3-sphinx-rtd-theme <!nodoc>,
33 python3-tifffile <!nodoc>,
34 python3-ujson,
35 python3-xarray,
36 python3-zarr
37 Standards-Version: 4.7.4
38 Testsuite: autopkgtest-pkg-pybuild
39 Homepage: https://github.com/fsspec/kerchunk
40 Vcs-Browser: https://salsa.debian.org/debian-gis-team/kerchunk
41 Vcs-Git: https://salsa.debian.org/debian-gis-team/kerchunk.git
42 Description: Cloud-friendly access to archival data
43 Kerchunk is a library that provides a unified way to represent a
44 variety of chunked, compressed data formats (e.g. NetCDF, HDF5, GRIB),
45 allowing efficient access to the data from traditional file systems or
46 cloud object storage. It also provides a flexible way to create
47 virtual datasets from multiple files. It does this by extracting the
48 byte ranges, compression information and other information about the
49 data and storing this metadata in a new, separate object.
50 This means that you can create a virtual aggregate dataset over
51 potentially many source files, for efficient, parallel and
52 cloud-friendly *in-situ* access without having to copy or translate
53 the originals. It is a gateway to in-the-cloud massive data processing
54 while the data providers still insist on using legacy formats for
55 archival storage.
56 .
57 Features:
58 .
59 * completely serverless architecture
60 * metadata consolidation, so you can understand a many-file dataset
61 (metadata plus physical storage) in a single read
62 * read from all of the storage backends supported by fsspec,
63 including object storage (s3, gcs, abfs, alibaba), http, cloud user
64 storage (dropbox, gdrive) and network protocols (ftp, ssh, hdfs,
65 smb...)
66 * loading of various file types (currently netcdf4/HDF, grib2, tiff,
67 fits, zarr), potentially heterogeneous within a single dataset,
68 without a need to go via the specific driver (e.g., no need for
69 h5py)
70 * asynchronous concurrent fetch of many data chunks in one go,
71 amortizing the cost of latency
72 * parallel access with a library like zarr without any locks
73 * logical datasets viewing many (>~millions) data files, and direct
74 access/subselection to them via coordinate indexing across an
75 arbitrary number of dimensions
76
77 Package: python3-kerchunk
78 Architecture: all
79 Depends: ${python3:Depends},
80 ${misc:Depends}
81 Recommends: hdf5-filter-plugin,
82 python3-cfgrib,
83 python3-cftime,
84 python3-h5py,
85 python3-hdf5plugin,
86 python3-scipy,
87 python3-xarray
88 Suggests: bitshuffle,
89 python3-aiohttp,
90 python3-dask,
91 python3-netcdf4,
92 python3-s3fs
93 Description: ${source:Synopsis}
94 ${source:Extended-Description}
95
96 Package: python-kerchunk-doc
97 Section: doc
98 Architecture: all
99 Multi-Arch: foreign
100 Depends: ${sphinxdoc:Depends},
101 ${misc:Depends}
102 Suggests: www-browser
103 Description: ${source:Synopsis} (documentation)
104 ${source:Extended-Description}
105 .
106 This package provides the HTML documentation for kerchunk.