Bug #79213 [Ver]: mb_check_encoding doesn't return true with valid base64 padded string

Edit report at https://bugs.php.net/bug.php?id=79213&edit=1 ID: 79213 Updated by: requinix@php.net Reported by: info at ioweb dot gr Summary: mb_check_encoding doesn't return true with valid base64 padded string Status: Verified Type: Bug Package: mbstring related Operating System: Debian 9, Ubuntu 18.04 PHP Version: 7.2.27 Block user comment: N Private report: N New Comment: @info: Yes, I was wrong about the nature of the bug, and the code I checked my response with also happened to be wrong (and in a way that made it look like mb_check_encoding was working). You can disregard most of what I said :( Previous Comments: ------------------------------------------------------------------------ [2020-02-03 05:15:38] info at ioweb dot gr I'd like to mention that For word "δοκιμή" the encoded string is zrTOv866zrnOvM6u and the result is false as well. It's not the padded = signs only. ------------------------------------------------------------------------ [2020-02-02 22:50:06] requinix@php.net Ah shoot, you're right. It's mb_detect_encoding that works on partial strings, not mb_check_encoding. ------------------------------------------------------------------------ [2020-02-02 22:43:39] nikic@php.net > However, I don't think that makes this behavior correct. If it were reading a UTF-8 stream > and the string ended in the middle of a multibyte sequence then it should report as valid. No, this is not how mb_check_encoding() is supposed to work. The string as a whole must be valid, not just a valid prefix. (Unfortunately many flush implementations in mbfl are somewhat broken, so theory and practice may not align very well.) ------------------------------------------------------------------------ [2020-02-02 22:31:36] requinix@php.net I believe the issue is that mb_check_encoding() works on the assumption the input comes from a stream, meaning there may be more input to read after what was passed to it. "test" encodes to "dGVzdA==" which is a valid base-64 string, however =s are strictly used as padding at the end of the output and so cannot appear in the middle of a stream. As such, if you rtrim the =s off then mb_check_encoding will report the "dGVzdA" is valid. But then that puts the responsibility of validating the number of =s on the user. However, I don't think that makes this behavior correct. If it were reading a UTF-8 stream and the string ended in the middle of a multibyte sequence then it should report as valid: there may be more to read that would complete the sequence. Valid "so far". The case of a padded base-64 string is different as the string ended in a valid state - it would be invalid if more bytes followed, but failing because there *might* be more isn't good. Would be nice if ext/mbstring provided a true stateful validator. Like a class that you feed stream input as it arrives and can query for the current validation state, or a procedural function that returns some state identifier that you pass back to it with subsequent calls. ------------------------------------------------------------------------ [2020-02-02 22:13:48] info at ioweb dot gr From reading the documentation I reached this assumption In the docs it says for mb_check_encoding Checks if the specified byte stream is valid for the specified encoding. A list of supported encodings I can get from mb_list_encodings which will also include base64 So I would assume mb_check_encoding would be able to handle all encodings that mb_list_encodings shows ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=79213 -- Edit this bug report at https://bugs.php.net/bug.php?id=79213&edit=1