[php-src] Issue #8308: The return value of mb_list_encodings() doesn't contain 'SJIS-win'

From: Date: Thu, 07 Apr 2022 21:14:42 +0000
Subject: [php-src] Issue #8308: The return value of mb_list_encodings() doesn't contain 'SJIS-win'
Groups: php.bugs 
Request: Send a blank email to php-bugs+get-240780@lists.php.net to get a copy of this message
Issue: https://github.com/php/php-src/issues/8308 Comment Author: sj-i OK, I don't know if this will fit nicely into the code comments, but let me explain my findings. In my opinion, unfortunately, the two encodings are often chosen almost at random among users by coincidence or historical reasons, rather than by strong opinions. As @zonuexe said, SJIS-win existed at the time of the first commit of libmbfl. libmbfl's predecessor was mbfilter, a program created by a Japanese company called HappySize and released as OSS for use with PHP. Although the implementation history at the time of the introduction of SJIS-win cannot be tracked anymore, what is important to this story is the initial implementation of libmbfl used with PHP's mbstring. On August 23, 2003, ***SJIS-win and CP932 were aliases***. The filename of the filter was mbfilter_cp932.c. At this point, U+00A5 and U+203E were mapped to FULLWIDTH YEN SIGN and FULLWIDTH MACRON, respectively. https://github.com/moriyoshi/libmbfl/commit/6c6ea25dd04aecc4260f0231b8158cc36b5e66a3#diff-5eecf532cc3c80d6877f1356bbc4b084ccd049313ab175160a1f9370384d1362R59-R63 https://github.com/moriyoshi/libmbfl/commit/6c6ea25dd04aecc4260f0231b8158cc36b5e66a3#diff-5eecf532cc3c80d6877f1356bbc4b084ccd049313ab175160a1f9370384d1362R237-R240 On March 14, 2004, A statement was added to the PHP Manual https://github.com/php/doc-en/commit/cee35d96cdddb02c6d6cb627f2c7dedcb2638e7a#diff-d5b0e3b6642a9b9829d0fc664170f19ddb344fd869a5317934d22db6e8bf179cR346 > For the CP932 codemap, use SJIS-WIN instead. This phrase is [still in the PHP manual](https://www.php.net/manual/en/mbstring.encodings.php), so the number of users who refer to the PHP manual to use CP932 would specify 'SJIS-win' because of this. On the other hand, Japanese programmers often know that there are multiple variants of ShiftJIS, and that some characters can only be handled with a character code called CP932 or Windows-31J. Some people may have "tried" one of them and it worked. At this point, these were just aliases, so the behavior was the same no matter what name they specified. They could use any name they like. On March 2, 2010, a couple of fixes were made to libmbfl that changes the conversion of U+00A5 and U+203E to 0x5c and 0x7e for CP932, respectively, and separate SJIS-win as a compatible implementation. https://github.com/moriyoshi/libmbfl/commit/cb257bc380e9d93865590dfb8593c3524a9b48a6 https://github.com/moriyoshi/libmbfl/commit/46a83aafb0738e49a5479769b3f649ac45eab087 For the first 10 years after the release of PHP 5.3.3 on July 22, 2010, and until the release of PHP 8.0, CP932 and SJIS-win will have different conversion tables for the two characters. And this difference is not specifically explained in the PHP manual. During this decade, many users may have unwittingly written programs that rely on a conversion table that they chose for no particular reason. On October 30, 2011, another page in the PHP manual also specifically lists CP932 as a supported encoding. https://github.com/php/doc-en/commit/93c8f3c1305ca9d17a6e9f68d9e0438dec5dc165 Depending on perspective, when users try to find out from the manual how to use CP932 with PHP, the PHP manual itself now supports two factions, CP932 and SJIS-win, incites warfare between them^^ Of course, there would be some programmers who will notice the difference between the two starting with PHP 5.3.3 and consciously choosing the encoding. Yes, don't forget about this faction: ....... The more serious and careful programmers know that there is no single right answer to the code conversion rules and that [they should be used differently for different purposes](https://web.archive.org/web/20120210172435/http://home.m05.itscom.net/numa/cde/ucs-conv/ucs-conv.html).

« previous php.bugs (#240780) next »