Re: utf-8 filenames in phar files.

On Mon, Apr 21, 2014 at 11:44 AM, Jakub Zelenka <bukka@php.net> wrote: > On Sat, Feb 15, 2014 at 10:53 AM, Yasuo Ohgaki <yohgaki@ohgaki.net> wrote: > >> On Sat, Feb 15, 2014 at 7:50 PM, Lester Caine <lester@lsces.co.uk> wrote: >> >> > My previous post did not appear on the list ;) >> > >> > >> > Yasuo Ohgaki wrote: >> > >> >> A lot of the current confusion does seem to be based around the >> >> Windows >> >> Wide-API as documented in 'The Problem' section of that document. >> It >> >> would >> >> seem that my 'naive' view of simply using UTF-8 strings is thwarted >> >> by these >> >> problems?-- >> >> >> >> Unicode is like one name with several encoding. We cannot get away from >> >> conversions, normalization especially. >> >> >> > >> > That is why personally I'm just looking at UTF8. Which is enough of a >> mine >> > field on it's own, but since a large swath of what we are working with >> now >> > is only UTF8 it does seem to be the right base going forward? >> > >> >> I have problem, too. It seems someone is working on DKIM. >> Anyway, there are problems, but UTF-8 is way to go. We just cannot remove >> conversions. Normalization has number of issues, including security >> related >> one. >> >> Regards, >> >> -- >> Yasuo Ohgaki >> yohgaki@ohgaki.net >> > > > Hi Stas, > > I just saw that the patch has been merged. There was an objection from > Yasuo about normalization in past... > > What's worse the patch doesn't check UTF-8 correctly. It accepts invalid > UTF-8. The correct spec is in > https://tools.ietf.org/html/rfc3629#section-4 . I have got re2c > implementation in jsond scanner : > > https://github.com/bukka/php-jsond/blob/master/jsond_scanner.re#L122-L134. > As you can see it's very different from the provided impl. > > I think that accepting ill-formed UTF-8 would be a mistake and as such the > patch should be reverted. > > Thanks > > Jakub > > I have created a quick PR: https://github.com/php/php-src/pull/649 that is fixing the ill-formed UTF-8 paths. Jakub