Re: Re: ZTS improvement idea

On Fri, 15 Feb 2019 at 07:26, Benjamin Eberlei <kontakt@beberlei.de> wrote: > This is like the 80% use-case of threading, Multiple HTTP requests, multiple long running SQL > queries. An API like parallel would allow each and everyone of us to make controllers faster today > without a large effort. > > I think an expanded example would better demonstrate the power of this. Consider a search API that aggregates other search APIs; for each API, it will need to do several steps: - Look up code mappings in a DB to convert input into format needed by this API - Perform an auth request to get a fresh token - Send the search API call - Look up more code mappings to convert the response - Post-process results into a standard form The challenge is to call as many APIs as possible at once, and get the results to the user. The slow parts of this are mostly I/O bound, but to take advantage of async I/O you need to rewrite not just the I/O parts but the entire process to use some async framework. Maybe there's even parts that are CPU heavy in between the requests - a complex cryptographic algorithm, for instance - that async I/O wouldn't help with at all, but which could happen on separate cores when load was low. It would be absolutely amazing if you could use something like Joe's new parallel extension to handle this workload: run the handling of each API as a thread, and leave the code inside them entirely unchanged. I realise it's probably not as simple as that in practice, but to me, the ability to adapt existing code is a huge benefit of a straightforward parallel/thread/worker API over one based explicitly on async I/O. Regards, -- Rowan Collins [IMSoP]