Bug #55477 [Com]: crypt() returns inconsistent hashes for non-ASCII characters

Edit report at https://bugs.php.net/bug.php?id=55477&edit=1 ID: 55477 Comment by: solar at openwall dot com Reported by: christian at pingdom dot com Summary: crypt() returns inconsistent hashes for non-ASCII characters Status: Bogus Type: Bug Package: *Encryption and hash functions Operating System: Linux PHP Version: 5.3.7 Block user comment: N Private report: N New Comment: Oops, I did not realize this interface would use <pre>. Somehow the preview does not. I think it's my first time posting a comment to a PHP bug. Let me try again, with line wrapping at 79 for hopefully easier reading: This is definitely the expected behavior, but saying that $2x$ should be used for non-ASCII is not entirely correct (and is a dangerous thing to say). If it were this simple, we'd do it in PHP itself, but we had good reasons not to. More specifically: The change as implemented in PHP 5.3.7+ favors security and correctness over backwards compatibility, but it also lets users (admins of PHP app installs) use the new $2x$ prefix on existing hashes to preserve backwards compatibility for those and incur the associated security risk until all such passwords are changed (using $2a$ or $2y$ for newly changed passwords). In versions of PHP older than 5.3.7, $2a$ inadvertently resulted in system-specific behavior for passwords with non-ASCII characters in them. On some installs (mostly on PowerPC and ARM, as well as sometimes on *BSD's and Solaris regardless of CPU architecture), they were processed correctly. On most installs (most Linux, many others), they were processed incorrectly most of the time (but not always), and moreover in a way where security was weakened. In PHP 5.3.7, $2a$ results in almost the correct behavior, but with an additional countermeasure against security-weakened old hashes mentioned above. $2x$ results in the buggy behavior, so if old hashes are known to be of the buggy type, this may be used on them to keep them working, accepting the associated security risk. $2y$ results in perfectly correct behavior (without the countermeasure), so it may be used (if desired) when hashing newly set passwords. For practical purposes, it does not really matter if you use $2a$ or $2y$ for newly set passwords, as the countermeasure is only triggered on some obscure passwords (not even valid UTF-8) that are unlikely to be seen outside of a deliberate attack (trying to match hashes produced by buggy pre-5.3.7 code). BTW, PHP 5.3.7+ has been updated to crypt_blowfish 1.2, not the intermediate 1.1 release referenced in the previous comment. The differences between 1.1 and 1.2 include introduction of the countermeasure for $2a$ mentioned above and the $2y$ prefix. Summary: for passwords without characters with the 8th bit set, there's no issue, all three prefixes work exactly the same. For occasional passwords with characters with the 8th bit set, if the app prefers security and correctness over backwards compatibility, no action is needed - just upgrade to new PHP and use its new behavior (with $2a$). However, if an app install admin truly prefers backwards compatibility over security, and the problem is seen on the specific install (which is not always the case because not all platforms/builds were affected), then $2a$ in existing hashes in the database may be changed to $2x$. Alternatively, a similar thing may be achieved by changing $2a$ to $2x$ in PHP app code after database queries, and using $2y$ on newly set passwords (such that the app's automatic change to $2x$ on queries is not triggered for them). Previous Comments: ------------------------------------------------------------------------ [2011-08-22 15:21:31] solar at openwall dot com This is definitely the expected behavior, but saying that $2x$ should be used for non-ASCII is not entirely correct (and is a dangerous thing to say). If it were this simple, we'd do it in PHP itself, but we had good reasons not to. More specifically: The change as implemented in PHP 5.3.7+ favors security and correctness over backwards compatibility, but it also lets users (admins of PHP app installs) use the new $2x$ prefix on existing hashes to preserve backwards compatibility for those and incur the associated security risk until all such passwords are changed (using $2a$ or $2y$ for newly changed passwords). In versions of PHP older than 5.3.7, $2a$ inadvertently resulted in system-specific behavior for passwords with non-ASCII characters in them. On some installs (mostly on PowerPC and ARM, as well as sometimes on *BSD's and Solaris regardless of CPU architecture), they were processed correctly. On most installs (most Linux, many others), they were processed incorrectly most of the time (but not always), and moreover in a way where security was weakened. In PHP 5.3.7, $2a$ results in almost the correct behavior, but with an additional countermeasure against security-weakened old hashes mentioned above. $2x$ results in the buggy behavior, so if old hashes are known to be of the buggy type, this may be used on them to keep them working, accepting the associated security risk. $2y$ results in perfectly correct behavior (without the countermeasure), so it may be used (if desired) when hashing newly set passwords. For practical purposes, it does not really matter if you use $2a$ or $2y$ for newly set passwords, as the countermeasure is only triggered on some obscure passwords (not even valid UTF-8) that are unlikely to be seen outside of a deliberate attack (trying to match hashes produced by buggy pre-5.3.7 code). BTW, PHP 5.3.7+ has been updated to crypt_blowfish 1.2, not the intermediate 1.1 release referenced in the previous comment. The differences between 1.1 and 1.2 include introduction of the countermeasure for $2a$ mentioned above and the $2y$ prefix. Summary: for passwords without characters with the 8th bit set, there's no issue, all three prefixes work exactly the same. For occasional passwords with characters with the 8th bit set, if the app prefers security and correctness over backwards compatibility, no action is needed - just upgrade to new PHP and use its new behavior (with $2a$). However, if an app install admin truly prefers backwards compatibility over security, and the problem is seen on the specific install (which is not always the case because not all platforms/builds were affected), then $2a$ in existing hashes in the database may be changed to $2x$. Alternatively, a similar thing may be achieved by changing $2a$ to $2x$ in PHP app code after database queries, and using $2y$ on newly set passwords (such that the app's automatic change to $2x$ on queries is not triggered for them). ------------------------------------------------------------------------ [2011-08-22 13:29:04] bjori@php.net This is expected, see http://www.openwall.com/lists/announce/2011/06/21/1 You need to use $2x$ for non-ascii, sorry :( ------------------------------------------------------------------------ [2011-08-22 12:47:41] christian at pingdom dot com Description: ------------ Hashes generated with crypt() (using Blowfish) on PHP 5.3.5 or 5.3.3 cannot be validated on 5.3.7, if the hashed strings contain non-ASCII characters. The reverse is also true, if the hashes were generated on 5.3.7, they cannot be validated on 5.3.3 or 5.3.5. Test script: --------------- $passwords = array( // these hashes were generated on PHP 5.3.5-1ubuntu7.2 with Suhosin-Patch (cli) (built: May 2 2011 23:00:17) 'brownfox' => '$2a$07$usesomesillystringforeD/hyr5e1bWX2PzwckMuCRNQMTrQNr72', 'Boxkämpfer' => '$2a$07$usesomesillystringfore36pVDWFz65CbxoLgSgVURqHWU4yEqye', 'щастлива' => '$2a$07$usesomesillystringforeoM7K1pyDjeAG1F42k34MP.tbiMnNcy.', 'Põdur' => '$2a$07$usesomesillystringfore1iPxMN9wh4Cr2oVR6nmdILWylX9D0iO', ); foreach ($passwords as $password => $hash) { $computedHash = crypt($password, $hash); if ($computedHash == $hash) { echo "hash OK\n"; } else { echo "hash FAIL ($hash != $computedHash)\n"; } } Expected result: ---------------- hash OK hash OK hash OK hash OK Actual result: -------------- hash OK hash FAIL ($2a$07$usesomesillystringfore36pVDWFz65CbxoLgSgVURqHWU4yEqye != $2a$07$usesomesillystringforeelZZJE6VQ2/DIcx1J.D.htZuAQIV43S) hash FAIL ($2a$07$usesomesillystringforeoM7K1pyDjeAG1F42k34MP.tbiMnNcy. != $2a$07$usesomesillystringforevg24bYcXKv2WUiCZvAH627ba6aubiNC) hash FAIL ($2a$07$usesomesillystringfore1iPxMN9wh4Cr2oVR6nmdILWylX9D0iO != $2a$07$usesomesillystringforeuqJNc6ZnvGzLGss/.ZcwQdygkbYamRq) ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=55477&edit=1