Re: [php6] Unicode support, options?

On Fri, Feb 21, 2014 at 1:04 PM, Lester Caine <lester@lsces.co.uk> wrote: > Pierre Joye wrote: >>> >>> What do you understand by "storage"? >> >> To have string stored as UTF-8 only, no conversion required for 99% of our >> use. > > > I think that the first thing that needs to be agreed on is if there will be > support for UTF-8 in the core? As has already been said, in many places this > currently just works and so blocking that may be more of a problem now? The > question surly is "What is the 1% that needs some extra work?" I think we pretty much agree already that we need UTF-8 as the base, meaning are stored in UTF-8. Conversions may be needed for advanced usages provided by ICU (or maybe not, I just do not know for sure now). > I light library would be most appropriate for filling the gaps currently > created by use of UTF-8 strings in the core? It is not until one starts > adding the mbstring level of string processing that a more powerful library > is required. Something that simply ensures UTF-8 strings are valid and can > carry out comparisons as required? it is more than only comparison. If only comparison, additions and the likes, utf8proc is enough, or librope with some additions. > The black hole is still 'case sensitivity' and it is perhaps laying down a > 'light' set of rules for this which would allow a path forward? As I have > indicated, I'd prefer simply dropping case insensitivity, but a compromise > might be to retain it where a string length does not change, and a clean > reverse transform exists? So a library that provides that comparison as part > of the core package? I do not care much about languages support for UTF-8 names for methods, functons, variables etc. My take on it is that we should stick to ASCII for it and be done with that. But that's only my opinion :) We may end writing our own library for the core operations... But I would prefer to avoid that as it is really not a trivial task. Cheers, -- Pierre @pierrejoye | http://www.libgd.org