Bug #67276 [Opn]: mb_strwidth count combining chars

Edit report at https://bugs.php.net/bug.php?id=67276&edit=1 ID: 67276 Updated by: cmb@php.net Reported by: nicolas dot grekas+php at gmail dot com Summary: mb_strwidth count combining chars Status: Open Type: Bug Package: mbstring related PHP Version: 5.5.12 Block user comment: N Private report: N New Comment: > Unless I'm missing something, it doesn't look like these > actually specify how the functions are supposed to work... Well, wcwidth() accepts a wchar_t, and to my knowledge, that would be a single code point for Unicode encodings, so no grapheme cluster support. Nikolas may have referred to some other standard. Previous Comments: ------------------------------------------------------------------------ [2021-08-16 16:46:49] nikic@php.net > Likely > <https://pubs.opengroup.org/onlinepubs/9699919799/functions/wcwidth.html> > and > <https://pubs.opengroup.org/onlinepubs/9699919799/functions/wcswidth.html>. Unless I'm missing something, it doesn't look like these actually specify how the functions are supposed to work... ------------------------------------------------------------------------ [2021-08-16 16:19:19] cmb@php.net > You mentioned that there is a standard for this, could you point > me to it? Likely <https://pubs.opengroup.org/onlinepubs/9699919799/functions/wcwidth.html> and <https://pubs.opengroup.org/onlinepubs/9699919799/functions/wcswidth.html>. Note, though, that <https://unicode-org.atlassian.net/browse/ICU-12726> mentions that these implementations were out of date/limited. Not sure about the linked implementation. ------------------------------------------------------------------------ [2021-08-16 16:13:04] nikic@php.net At least going by the referenced C implementation, this looks a lot simpler than grapheme clusters, so it's something we might include. You mentioned that there is a standard for this, could you point me to it? I know about UAX#11, but I don't think that specifies an actual character width algorithm. ------------------------------------------------------------------------ [2021-08-16 15:34:19] cmb@php.net All of MBString is broken with regard to grapheme clusters; these are not catered to by other MBString functions as well[1]. I don't think this will ever change. It might make sense to introduce grapheme_strwidth(), but apparently ICU does not support that yet[2]. I'd rather avoid an own implementation (or relying on wcwidth()), since these may easily become outdated regarding new Unicode features. Not sure what to do here. Maybe you want to write to internals? [1] <https://3v4l.org/taWVn> [2] <https://unicode-org.atlassian.net/browse/ICU-12726> ------------------------------------------------------------------------ [2021-08-16 15:23:14] nicolasgrekas@php.net But then, the function is just broken for the purpose it should serve. For reference, there is standard on the topic, which is implemented in C at: https://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c and in Python at https://github.com/jquast/wcwidth. In PHP, there is this: https://github.com/symfony/string/blob/bd53358e3eccec6a670b5f33ab680d8dbe1d4ae1/AbstractUnicodeString.php#L508 ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=67276 -- Edit this bug report at https://bugs.php.net/bug.php?id=67276&edit=1