Bug->Doc #63663 [Ana]: str_word_count does not properly handle non-latin characters

Edit report at https://bugs.php.net/bug.php?id=63663&edit=1 ID: 63663 Updated by: cmb@php.net Reported by: kobrien at kiva dot org Summary: str_word_count does not properly handle non-latin characters Status: Analyzed -Type: Bug +Type: Documentation Problem Package: Strings related Operating System: Ubuntu 12.04 PHP Version: 5.3.20-dev -Assigned To: +Assigned To: cmb Block user comment: N Private report: N New Comment: > […] it's something we'd need to think about as part of making > the whole language more multibyte-aware. That's unlikely going to happen, so I'm changing to doc bug. Previous Comments: ------------------------------------------------------------------------ [2015-05-08 23:11:54] cmb@php.net Related To: Bug #65080 ------------------------------------------------------------------------ [2015-04-12 20:28:38] cmb@php.net Related To: Bug #27668 ------------------------------------------------------------------------ [2012-12-03 03:10:21] kobrien at kiva dot org Ok feature request filed here: https://bugs.php.net/bug.php?id=63671 First time doing that, so hopefully it's correctly filed. ------------------------------------------------------------------------ [2012-12-03 02:47:28] aharvey@php.net Yeah, a feature request for mb_str_word_count() might be a good idea. The isalpha() issue isn't really PHP specific: the underlying C function simply takes a single byte as its input, so it can't ascertain whether a multibyte character is actually alphanumeric or not (since it only ever gets the first byte of the sequence). There's an iswalpha() function that would do the right thing, but PHP was written before it was widely available, and using it in str_word_count() alone would be inconsistent with the rest of the language: it's something we'd need to think about as part of making the whole language more multibyte-aware. ------------------------------------------------------------------------ [2012-12-03 02:36:37] kobrien at kiva dot org Thanks for the reply. Given your comments about the problems, would it be helpful for me to also file a feature request for newer versions of php to have a mb_str_word_count function which could properly handle this case? I haven't dug into the C code enough to understand why isalpha() fails on multibyte, but I'd have to imagine there is an alternative available that will handle multi-byte characters properly. I could potentially even create a patch if pointed in the right direction. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at https://bugs.php.net/bug.php?id=63663 -- Edit this bug report at https://bugs.php.net/bug.php?id=63663&edit=1