Re: What "Unicode support" really means

On Thu, Mar 6, 2014 at 9:59 AM, Lester Caine <lester@lsces.co.uk> wrote: > There are simply two types of string handling that we need to worry about. > Simply comparing case sensitive names UTF-8 just works and there is very > little internal work to do? It's exactly which path is used to process case > comparison which affects internal text strings that is the sticking point? > Coming into this 'area' is ordering of strings, which is where a particular > locale becomes important. But just what operations does that actually > effect? Is it just a process that comes as part of the display conversion or > is it more deeply embedded in the 'core'? Personally I already handle this > via the database, since I only want the correct subset of data to display, > but that is not a generic solution :( I do not understand the points behind these questions, sorry... > The second one wants to carry out more 'character' related processing, which > IS currently relegated to mbstring ( or again perhaps the database engine? ) > but that does not affect the bulk of program flow? I'm perfectly happy that > at this level of processing the switch is to UTF-32 which also seems to be > the general direction the ICU is developing? Let mbstring on the side for now, it is not part of the possible options as far as I can see. ICU, and please read the ICU documentation, it will help a lot to do not have to explain these things :), uses UTF-16 by default for its internal storage. > I do not seem to be getting an answer to the one bit I can't get through my > own limited delving into the code. Just what 'character handling' operations > that need to be codepoint aware are required for the 'program' operations > rather than simply handling 'data' strings? split, upper/lower case operations, word splitting, pcre or actually almost all string operations need to be CP aware. However I am not convinced that we have to deal with the existing function at this stage, it is good to discuss the issue but I think the design of the unicode support implementation is by far more important. Prototyping can be done using new APIs. Existing APIs can be more easily ported later, if desired. Cheers, -- Pierre @pierrejoye | http://www.libgd.org