Bug #63316 [Opn->Nab]: Files declared as UTF-8 will append NUL bytes to output for non ASCII character

From: Date: Mon, 25 Jul 2016 16:56:18 +0000
Subject: Bug #63316 [Opn->Nab]: Files declared as UTF-8 will append NUL bytes to output for non ASCII character
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-202585@lists.php.net to get a copy of this message
Edit report at https://bugs.php.net/bug.php?id=63316&edit=1

 ID:                 63316
 Updated by:         cmb@php.net
 Reported by:        post at wickenrode dot com
 Summary:            Files declared as UTF-8 will append NUL bytes to
                     output for non ASCII character
-Status:             Open
+Status:             Not a bug
 Type:               Bug
 Package:            Unicode Engine related
 Operating System:   Xubuntu 12.10 amd64
 PHP Version:        5.4.8
-Assigned To:        
+Assigned To:        cmb
 Block user comment: N
 Private report:     N

 New Comment:

> set 
>  mbstring.internal_encoding = utf-8
> in php.ini, and then everything works fine.

That. See also
<http://php.net/manual/en/ini.core.php#ini.zend.script-encoding>.


Previous Comments:
------------------------------------------------------------------------
[2013-02-17 07:16:44] wynn dot chen dot cn at gmail dot com

i think it's not a bug, just something like #62351.
set 
  mbstring.internal_encoding = utf-8
in php.ini, and then everything works fine.

------------------------------------------------------------------------
[2012-10-19 21:56:47] post at wickenrode dot com

Description:
------------
Take the test script below and save it as UTF-8 and verify:

> cat test.php | tail -n 4 | head -n 1 | hexdump -C
00000000  09 23 20 c3 a4 c3 b6 0a                           |.# .....|
00000008

Now run

> ./bin/php test.php | hexdump -C
PHP Version = 5.4.8
zend.multibyte = 1
00000000  58 00 00 00                                       |X...|
00000004

As you can see PHP added three NUL bytes to the file.
Now remove one of the characters in the commented out line:

> ./bin/php test.php | hexdump -C
PHP Version = 5.4.8
zend.multibyte = 1
00000000  58 00 00                                          |X..|
00000003

As you can see the NUL bytes are directly related to the amount of non-ASCII characters in the file.

Now comment out the declare statement:

> ./bin/php test.php | hexdump -C
PHP Version = 5.4.8
zend.multibyte = 1
00000000  58                                                |X|
00000001

This time the output is correct.

Tested with off-the-shelf PHP, but also happens with ubuntu packaged 5.4.6-1
Also happends when PHP is running as Apache module where this issue caused corrupted images for me.

Test script:
---------------
<?php
	declare(encoding='UTF-8');

	fwrite(STDERR,"PHP Version = ".phpversion()."\n");
	fwrite(STDERR,"zend.multibyte = ".ini_get('zend.multibyte')."\n");

	# äöü
	echo "X";

?>


Expected result:
----------------
See above.
UTF-8 declared files with UTF-8 content should work fine.

Actual result:
--------------
See above.
UTF-8 declared files with UTF-8 content produce NUL bytes in the output.


------------------------------------------------------------------------



--
Edit this bug report at https://bugs.php.net/bug.php?id=63316&edit=1


Thread (3 messages)

« previous php.bugs (#202585) next »