| 1 |
liberasurecode |
| 2 |
============== |
| 3 |
|
| 4 |
liberasurecode is an Erasure Code API library written in C with pluggable Erasure Code backends. |
| 5 |
|
| 6 |
---- |
| 7 |
|
| 8 |
Highlights |
| 9 |
========== |
| 10 |
|
| 11 |
* Unified Erasure Coding interface for common storage workloads. |
| 12 |
|
| 13 |
* Pluggable Erasure Code backends - liberasurecode supports the following backends: |
| 14 |
|
| 15 |
- 'liberasurecode_rs_vand' - Native, software-only Erasure Coding implementation that supports a Reed-Solomon backend |
| 16 |
- 'Jerasure' - Erasure Coding library that supports Reed-Solomon, Cauchy backends [1] |
| 17 |
- 'ISA-L' - Intel Storage Acceleration Library - SIMD accelerated Erasure Coding backends [2] |
| 18 |
- 'SHSS' - NTT Lab Japan's hybrid Erasure Coding backend [4] |
| 19 |
- 'Flat XOR HD' - built-in to liberasurecode, based on [3] |
| 20 |
- 'libphazr' - Phazr.IO's erasure code backend with built-in privacy [5] |
| 21 |
- 'NULL' template backend implemented to help future backend writers |
| 22 |
|
| 23 |
|
| 24 |
* True 'plugin' architecture - liberasurecode uses Dynamically Loaded (DL) |
| 25 |
libraries to realize a true 'plugin' architecture. This also allows one to |
| 26 |
build liberasurecode indepdendent of the Erasure Code backend libraries. |
| 27 |
|
| 28 |
* Cross-platform - liberasurecode is known to work on Linux (Fedora/Debian |
| 29 |
flavors), Solaris, BSD and Darwin/Mac OS X. |
| 30 |
|
| 31 |
* Community support - Developed alongside Erasure Code authority Kevin |
| 32 |
Greenan, liberasurecode is an actively maintained open-source project with |
| 33 |
growing community involvement (Openstack Swift, Ceph, PyECLib, NTT Labs). |
| 34 |
|
| 35 |
---- |
| 36 |
|
| 37 |
Active Users |
| 38 |
==================== |
| 39 |
|
| 40 |
* PyECLib - Python EC library: https://github.com/openstack/pyeclib |
| 41 |
* Openstack Swift Object Store - https://wiki.openstack.org/wiki/Swift |
| 42 |
|
| 43 |
|
| 44 |
---- |
| 45 |
|
| 46 |
Build and Install |
| 47 |
================= |
| 48 |
|
| 49 |
Install dependencies - |
| 50 |
|
| 51 |
Debian/Ubuntu hosts: |
| 52 |
|
| 53 |
```sh |
| 54 |
$ sudo apt-get install build-essential autoconf automake libtool |
| 55 |
``` |
| 56 |
|
| 57 |
Fedora/RedHat/CentOS hosts: |
| 58 |
|
| 59 |
```sh |
| 60 |
$ sudo yum install -y gcc make autoconf automake libtool |
| 61 |
``` |
| 62 |
|
| 63 |
To build the liberasurecode repository, perform the following from the |
| 64 |
top-level directory: |
| 65 |
|
| 66 |
``` sh |
| 67 |
$ ./autogen.sh |
| 68 |
$ ./configure |
| 69 |
$ make |
| 70 |
$ make test |
| 71 |
$ sudo make install |
| 72 |
``` |
| 73 |
|
| 74 |
---- |
| 75 |
|
| 76 |
Getting Help |
| 77 |
============ |
| 78 |
|
| 79 |
- Bugs: https://bugs.launchpad.net/liberasurecode/ |
| 80 |
- Mailing List: http://lists.openstack.org/pipermail/openstack-discuss/ (use tag `[swift][liberasurecode]`) |
| 81 |
- IRC: #openstack-swift on OFTC |
| 82 |
|
| 83 |
---- |
| 84 |
|
| 85 |
References |
| 86 |
========== |
| 87 |
|
| 88 |
[1] Jerasure, C library that supports erasure coding in storage applications, http://jerasure.org |
| 89 |
|
| 90 |
[2] Intel(R) Storage Acceleration Library (Open Source Version), https://01.org/intel%C2%AE-storage-acceleration-library-open-source-version |
| 91 |
|
| 92 |
[3] Greenan, Kevin M et al, "Flat XOR-based erasure codes in storage systems", http://www.kaymgee.com/Kevin_Greenan/Publications_files/greenan-msst10.pdf |
| 93 |
|
| 94 |
[4] Kota Tsuyuzaki <[email protected]>, "NTT SHSS Erasure Coding backend" |
| 95 |
|
| 96 |
[5] Jim Cheung <[email protected]>, "Phazr.IO libphazr erasure code backend with built-in privacy" |