Re: Smart strings & memory allocation

SS>> make test is synthetic. My primary use of smart_str involves SS>> IRCG based application. Which tells what? What are sizes you use there in IRCG? Anyway, is IRCG the most used client of smart strings? Since much more used printf functions use it too, I guess that with all due respect to IRCG it is not is not the primary user of this code. Thus, I don't think IRCG application is a good way to test system-wide function. But again, I am eager to see an example of general test that you think are better than make test. I chose make test as an example that uses a lot of different functions so my tests would not be aligned against one application's memory usage pattern. SS>> Most applications use smart_str for temporarily creating some SS>> representation, processing it quickly, and freeing it SS>> immediately afterwards. I.e. the overhead in memory Bingo. That's why caching it is so important - we can get rid of a lot of malloc() calls. If you can do one malloc instead of 1000 mallocs - that's very good in my book. But in the paragraph above I was talking about general [e]malloc, not about smart strings, though with smart strings it shows up too. SS>> Most malloc implementations are tuned for the repetitive SS>> malloc/free pattern of the same size. For those, using the SS>> engine's memory cache will not be an advantage, because it SS>> prevents memory from being reused through ordinary malloc. There's almost no 'ordinary malloc' in the engine. All mallocs go through emalloc. Actually, all mallocs should go through emalloc. Also, I would like more explanation about how exactly caching memory allocations can hurt performance - did you try to disable Zend memory cache and actually got better results that with it? Also, mallocs usually means multithreaded locks. Which, as I heard, is Very Bad (TM) for multithreaded performance. However, since ZE memory caches are per-thread, cached allocator does not need any locks. I know only one downside of the memory cache that can really hurt - it may create fragmentation in malloc arenas. That's why, as I stated in my last letter, I was thinking about separating most frequently allocated static blocks into separate fixed-block allocator. But this is a long way to test if it is actually important and profitable - I just started messing with it :) Still my question is unanswered - according to your data, does or does not the system malloc have some optimization to deal with small allocs? If I allocate 1024 times 20 bytes (meaning 1024 zvals - pretty reasonable number for a decent application) - would I get 128K memory footprint instead of 20K memory footprint? Because if yes - we are in a dire need for more effective allocator. -- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/ +972-3-6139665 ext.109