Properly working with binary data
| From: | Niklas Mollenhauer | Date: | Fri, 20 Jan 2023 12:39:07 +0000 |
| Subject: | Properly working with binary data | ||
| Groups: | php.general | ||
| Request: | Send a blank email to php-general+get-328134@lists.php.net to get a copy of this message | ||
Hello!
I'm currently trying to process binary data. I basically need something like an ArrayBuffer in
JavaScript [1] as well
as typed arrays [2] to read/write data and something like a DataView [3] with support for endianess
conversion.
The best approaches I know is to work with strings as a byte buffer as well as pack/unpack.
This gets slow if I want to write a Uint32 or some other data type at random positions. This is why
I think that
there must be some native extension for this (or an entirely different approach).
The current use case for my application also has a JS-based binary parser. It could be translated to
some PHP equivalent:
```php
// object backing the native array buffer
$buffer = new ArrayBuffer(4); // 4 bytes
// view on the array buffer (with write-access)
$view = new DataView($buffer);
// writing an int16 (2 bytes) at offset 0
$view->setInt16(0 /* byteOffset */, 256 /* value */, true /* littleEndian */);
// accessing the buffer on a byte-basis
$byteArray = new Uint8Array($buffer);
$byteArray[0]; // 0
$byteArray[1]; // 1
$byteArray[2]; // 0
$byteArray[3]; // 0
// accessing the buffer on a 16-bit basis
$int16Array = new Int16Array($buffer);
$int16Array[0]; // 256
$int16Array[1]; // 0
$int16Array[1] = 42;
$byteArray[0]; // 0
$byteArray[1]; // 1
$byteArray[2]; // 42
$byteArray[3]; // 0
$binaryData = $buffer->getBinaryString(); // traditional PHP string to work with
```
In some other use-case I tried implementing a BitSet/BitArray and used a string array as well as GMP
as backing buffers [4].
It turned out that the first approach is pretty inefficient and while GMP is fast for most
operations,
it is not reliable/consistent to ex/import binary buffers. This is not surprising to me, since GMP
doesn't seem to
be indented for binary data processing.
Is there something to handle this in a performant manner? If not, this would also make a good
candidate for a PHP
extension (or even a built-in extension). Mimicking the API of JS could be a viable option.
Regards,
Niklas
[1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/ArrayBuffer
[2]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/TypedArray
[3]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/DataView
[4]: https://github.com/nikeee/php-bit-array