Re: RFC: CachedIterable (rewindable, allows any key&repeating keys)

Hi Alex, > > I've created a new RFC > > https://wiki.php.net/rfc/cachediterable adding CachedIterable, > > which eagerly evaluates any iterable and contains an immutable copy of the keys and values > > of the iterable it was constructed from > > > > > > Any other feedback unrelated to namespaces? > > Hi Tyson, > > I needed this feature a few years ago. In that case, the source was a generator that was slowly > generating data while fetching them from a paginated API that had rate limits. > The result wrapping iterator was used at runtime in multiple (hundreds) other iterators that > were processing elements in various ways (technical analysis indicator on time series) and after > that merged back with some MultipleIterator. > > Just for reference, this is how the implementation in userland was and I was happy with it as a > solution: > > https://gist.github.com/drealecs/ad720b51219675a8f278b8534e99d7c7 > > Not sure if it's useful but I thought I should share it as I noticed you mentioned in your > example for PolyfillIterator you chose not to use an IteratorAggregate because complexity > Was wondering how much inefficient this would be compared to the C implementation. That was for simplicity(shortness) of the RFC for people reading the polyfill. I don't expect it to affect CPU timing or memory usage for large arrays in the polyfill. Userland lazy iterable implementations could still benefit from having a CachedIterable around, by replacing the lazy IteratorAggregate with a Cached Iterable when the end of iteration was detected. > Also, the implementation having the ability to be lazy was important and I think that should be > the case here as well, by design, especially as we are dealing with Generators. We're dealing with the entire family of iterables, including but not limited to Generators, arrays, user-defined Traversables, etc. I'd considered that but decided not to include it in the RFC's scope. If I was designing that, it would be a separate class LazyCachedIterable. Currently, CachedIterable has several useful properties: 1. Serialization/Unserializable behavior is predictable - if the object was constructed it can be safely serialized if keys/values can be serialized. 2. Iteration has no side effects (e.g. won't throw) 3. keyAt(int $offset) and so on have predictable behavior, good performance, and only one throwable type 4. Memory usage is small - this might also be the case for a LazyIterable depending on implementation choices/constraints. Adding lazy iteration support would make it no longer have some of those properties. While I'd be in favor of that if it was implemented correctly, I don't plan to work on implementing this until I know if the addition of CachedIterable to a large family of iterable classes would pass. CachedIterable has some immediate benefits on problems I was actively working on, such as: 1. Being able to represent iterable functions such as iterable_reverse() 2. Memory efficiency and time efficiency for iteration 3. Being something internal code could return for getIterator(), etc. Regards, Tyson