| 1 |
Source: bitshuffle |
| 2 |
Section: devel |
| 3 |
Priority: optional |
| 4 |
Maintainer: Debian PaN Maintainers <[email protected]> |
| 5 |
Uploaders: |
| 6 |
Debian Science Maintainers <[email protected]>, |
| 7 |
Picca Frédéric-Emmanuel <[email protected]>, |
| 8 |
Thorsten Alteholz <[email protected]>, |
| 9 |
Build-Depends: |
| 10 |
cython3, |
| 11 |
debhelper-compat (= 13), |
| 12 |
dh-exec, |
| 13 |
dh-python, |
| 14 |
dh-sequence-numpy3, |
| 15 |
dh-sequence-python3, |
| 16 |
hdf5-tools <!nocheck>, |
| 17 |
libhdf5-mpi-dev, |
| 18 |
libhdf5-serial-dev, |
| 19 |
liblz4-dev, |
| 20 |
liblzf-dev, |
| 21 |
libzstd-dev, |
| 22 |
mpi-default-bin, |
| 23 |
pkgconf, |
| 24 |
pybuild-plugin-pyproject, |
| 25 |
python3-all-dev, |
| 26 |
python3-h5py, |
| 27 |
python3-h5py-mpi, |
| 28 |
python3-numpy, |
| 29 |
python3-pytest <!nocheck>, |
| 30 |
python3-setuptools, |
| 31 |
Standards-Version: 4.6.2 |
| 32 |
Homepage: https://github.com/kiyo-masui/bitshuffle |
| 33 |
Vcs-Browser: https://salsa.debian.org/science-team/bitshuffle |
| 34 |
Vcs-Git: https://salsa.debian.org/science-team/bitshuffle.git |
| 35 |
Rules-Requires-Root: no |
| 36 |
|
| 37 |
Package: bitshuffle |
| 38 |
Section: libs |
| 39 |
Architecture: any |
| 40 |
Provides: |
| 41 |
hdf5-filter-plugin-bshuf-mpi, |
| 42 |
hdf5-filter-plugin-bshuf-openmpi, |
| 43 |
hdf5-filter-plugin-bshuf-serial, |
| 44 |
Depends: |
| 45 |
${misc:Depends}, |
| 46 |
${python3:Depends}, |
| 47 |
${shlibs:Depends}, |
| 48 |
Description: filter for improving compression of typed binary data |
| 49 |
Bitshuffle is an algorithm that rearranges typed, binary data for |
| 50 |
improving compression, as well as a python/C package that implements |
| 51 |
this algorithm within the Numpy framework. |
| 52 |
. |
| 53 |
The library can be used along side HDF5 to compress and decompress |
| 54 |
datasets and is integrated through the dynamically loaded filters |
| 55 |
framework. Bitshuffle is HDF5 filter number 32008. |
| 56 |
. |
| 57 |
Algorithmically, Bitshuffle is closely related to HDF5's Shuffle |
| 58 |
filter except it operates at the bit level instead of the byte level. |
| 59 |
Arranging a typed data array in to a matrix with the elements as the |
| 60 |
rows and the bits within the elements as the columns, Bitshuffle |
| 61 |
"transposes" the matrix, such that all the least-significant-bits |
| 62 |
are in a row, etc. This transpose is performed within blocks of |
| 63 |
data roughly 8kB long. |
| 64 |
. |
| 65 |
This does not in itself compress data, only rearranges it for more |
| 66 |
efficient compression. To perform the actual compression you will |
| 67 |
need a compression library. Bitshuffle has been designed to be well |
| 68 |
matched Marc Lehmann's LZF as well as LZ4. Note that because |
| 69 |
Bitshuffle modifies the data at the bit level, sophisticated entropy |
| 70 |
reducing compression libraries such as GZIP and BZIP are unlikely to |
| 71 |
achieve significantly better compression than simpler and faster |
| 72 |
duplicate-string-elimination algorithms such as LZF and LZ4. |
| 73 |
Bitshuffle thus includes routines (and HDF5 filter options) to apply |
| 74 |
LZ4 compression to each block after shuffling. |
| 75 |
. |
| 76 |
The Bitshuffle algorithm relies on neighbouring elements of a dataset |
| 77 |
being highly correlated to improve data compression. Any correlations |
| 78 |
that span at least 24 elements of the dataset may be exploited to |
| 79 |
improve compression. |