| 1 |
# clue/reactphp-utf8 |
| 2 |
|
| 3 |
[](https://github.com/clue/reactphp-utf8/actions) |
| 4 |
[](https://packagist.org/packages/clue/utf8-react) |
| 5 |
|
| 6 |
Streaming UTF-8 parser, built on top of [ReactPHP](https://reactphp.org/). |
| 7 |
|
| 8 |
**Table of Contents** |
| 9 |
|
| 10 |
* [Support us](#support-us) |
| 11 |
* [Usage](#usage) |
| 12 |
* [Sequencer](#sequencer) |
| 13 |
* [Install](#install) |
| 14 |
* [Tests](#tests) |
| 15 |
* [License](#license) |
| 16 |
* [More](#more) |
| 17 |
|
| 18 |
## Support us |
| 19 |
|
| 20 |
We invest a lot of time developing, maintaining and updating our awesome |
| 21 |
open-source projects. You can help us sustain this high-quality of our work by |
| 22 |
[becoming a sponsor on GitHub](https://github.com/sponsors/clue). Sponsors get |
| 23 |
numerous benefits in return, see our [sponsoring page](https://github.com/sponsors/clue) |
| 24 |
for details. |
| 25 |
|
| 26 |
Let's take these projects to the next level together! 🚀 |
| 27 |
|
| 28 |
## Usage |
| 29 |
|
| 30 |
### Sequencer |
| 31 |
|
| 32 |
The `Sequencer` class can be used to make sure you only get back complete, valid |
| 33 |
UTF-8 byte sequences when reading from a stream. |
| 34 |
It wraps a given `ReadableStreamInterface` and exposes its data through the same |
| 35 |
interface. |
| 36 |
|
| 37 |
```php |
| 38 |
<?php |
| 39 |
|
| 40 |
require __DIR__ . '/vendor/autoload.php'; |
| 41 |
|
| 42 |
$stdin = new ReadableResourceStream(STDIN); |
| 43 |
|
| 44 |
$stream = new Sequencer($stdin); |
| 45 |
|
| 46 |
$stream->on('data', function ($chunk) { |
| 47 |
var_dump($chunk); |
| 48 |
}); |
| 49 |
``` |
| 50 |
|
| 51 |
React's streams emit chunks of data strings and make no assumption about its encoding. |
| 52 |
These chunks do not necessarily represent complete UTF-8 byte sequences, as a |
| 53 |
sequence may be broken up into multiple chunks. |
| 54 |
This class reassembles these sequences by buffering incomplete ones. |
| 55 |
|
| 56 |
Also, if you're merely consuming a stream and you're not in control of producing and |
| 57 |
ensuring valid UTF-8 data, it may as well include invalid UTF-8 byte sequences. |
| 58 |
This class replaces any invalid bytes in the sequence with a `?`. |
| 59 |
This replacement character can be given as a second parameter to the constructor: |
| 60 |
|
| 61 |
```php |
| 62 |
$stream = new Sequencer($stdin, 'X'); |
| 63 |
``` |
| 64 |
|
| 65 |
As such, you can be sure you never get an invalid UTF-8 byte sequence out of |
| 66 |
the resulting stream. |
| 67 |
|
| 68 |
Note that the stream may still contain ASCII control characters or |
| 69 |
ANSI / VT100 control byte sequences, as they're valid UTF-8. |
| 70 |
This binary data will be left as-is, unless you filter this at a later stage. |
| 71 |
|
| 72 |
## Install |
| 73 |
|
| 74 |
The recommended way to install this library is [through Composer](https://getcomposer.org/). |
| 75 |
[New to Composer?](https://getcomposer.org/doc/00-intro.md) |
| 76 |
|
| 77 |
This project follows [SemVer](https://semver.org/). |
| 78 |
This will install the latest supported version: |
| 79 |
|
| 80 |
```bash |
| 81 |
composer require clue/utf8-react:^1.3 |
| 82 |
``` |
| 83 |
|
| 84 |
See also the [CHANGELOG](CHANGELOG.md) for details about version upgrades. |
| 85 |
|
| 86 |
This project aims to run on any platform and thus does not require any PHP |
| 87 |
extensions and supports running on legacy PHP 5.3 through current PHP 8+ and |
| 88 |
HHVM. |
| 89 |
It's *highly recommended to use the latest supported PHP version* for this project. |
| 90 |
|
| 91 |
## Tests |
| 92 |
|
| 93 |
To run the test suite, you first need to clone this repo and then install all |
| 94 |
dependencies [through Composer](https://getcomposer.org/): |
| 95 |
|
| 96 |
```bash |
| 97 |
composer install |
| 98 |
``` |
| 99 |
|
| 100 |
To run the test suite, go to the project root and run: |
| 101 |
|
| 102 |
```bash |
| 103 |
vendor/bin/phpunit |
| 104 |
``` |
| 105 |
|
| 106 |
## License |
| 107 |
|
| 108 |
This project is released under the permissive [MIT license](LICENSE). |
| 109 |
|
| 110 |
> Did you know that I offer custom development services and issuing invoices for |
| 111 |
sponsorships of releases and for contributions? Contact me (@clue) for details. |
| 112 |
|
| 113 |
## More |
| 114 |
|
| 115 |
* If you want to learn more about processing streams of data, refer to the documentation of |
| 116 |
the underlying [react/stream](https://github.com/reactphp/stream) component. |
| 117 |
|
| 118 |
* If you want to process ASCII control characters or ANSI / VT100 control byte sequences, you may |
| 119 |
want to use [clue/reactphp-term](https://github.com/clue/reactphp-term) on the raw input |
| 120 |
stream before passing the resulting stream to the UTF-8 sequencer. |
| 121 |
|
| 122 |
* If you want to to display or inspect the byte sequences, you may |
| 123 |
want to use [clue/hexdump](https://github.com/clue/php-hexdump) on the emitted byte sequences. |