Front page | perl.loop |
Postings from April 2008
Re: benchmarking various event loops with and without anyevent
Thread Previous
|
Thread Next
From:
Uri Guttman
Date:
April 26, 2008 02:34
Subject:
Re: benchmarking various event loops with and without anyevent
Message ID:
x7mynh6vvk.fsf@mail.sysarch.com
>>>>> "ML" == Marc Lehmann <schmorp@schmorp.de> writes:
ML> On Fri, Apr 25, 2008 at 11:40:03PM -0400, Uri Guttman <uri@stemsystems.com> wrote:
>> check out stem's pure perl event loop. there are examples in the
ML> Maybe I'll provide a backend for stem.
actually it makes more sense to me to wrap anyevent in stem. it already
has several event wrappers (pure perl, event.pm and tk) and wrapping is
very easy to do. not much different than the code i see in anyevent.
>> modules. it does things in a different direction and doesn't scan
>> select's bit masks but instead it scans the interesting handles and see
>> whether their bits are set. it should exhibit good behavior under growth
>> as all the data are managed in hashes.
ML> That is exactly as the anyevent perl backend did in the previous
ML> version. I don't see how that should exhibit good behaviour - file
ML> descristpors are small integers, so hashes only add overhead over arrays,
ML> and checking every file descriptor that way is much slower than scanning
ML> the bitmask.
ML> Especially in the important case of many handles/few active ones, an approach
ML> of "scan all handles" is very slow.
my code doesn't scan all handle, but scan all writeable events to see if
their handle bits are set. descriptors increment and can cause bloat of
the array if you have many in use (and not all of them in the event loop).
ML> (The benchmarks reflect this, you could try with an older anyevent
ML> release where we just check the keys of the hash storing fd
ML> information: performance is much lower).
ML> I then made a second benchmark, designed not to measure anyevent overhead,
ML> but to measure real-world performance of a socket server.
>>
>> and that /sessions code also shows use of the asyncio module. if you can
>> benchmark that i would be interested in the results.
ML> Hmm, tt seems to have become fashionable lately to call synchronous I/O
ML> asynchronous (see IO::Async, another 100% synchronous framework). What
ML> exactly is asynchronous about the I/O in that example, it seems to be
ML> fully synchronous to me (just event-driven).
event loops are async in that you get callbacks as they are needed. sure
you can't get true async behavior from any cpu/os combo but calling it
async is no different than calling it parallel processing when it is
just context switching among processes on a single cpu. the illusion and
api are what matters. a better term may be non-blocking i/o (and socket
connections) but that needs more explanation in some ways. my view is
that threads are the real sync app style and event loops are the async
style. but this is not the time nor place to discuss that.
>> i don't find that surprising. perl's i/o is decent and as i said above,
>> a loaded server is doing mostly i/o.
ML> Well, since the server in this benchmark hardly does anything, and as you
ML> can see by the results, the event loop completely dominates, and bad event
ML> loops are a thousand times slower than good ones.
i haven't looked at the benchmark stuff yet. i was browsing the code
earlier.
ML> When put into requests/s, this means that with POE, you are limited
ML> to <1000 requests/s, while with EV you can easily reach hundreds of
ML> thousands.
ML> Also, this does not explain at all why Event is so slow, and why Glib scales
ML> so extremely badly. Most of the stuff that slows down the perl-based event
ML> loop is clearly stuff that is much much faster in C.
poor memory management in the c code? i have done pure c event loops in
a framework where memory management was fairly fast due to cached queues
of known block sizes. alloc/free were usually a trivial call and the
event code had it own private fixed size buffer queues. it had no
problem doing 2k parallel web fetches including all the url parsing all
on a single sparc. we had to throttle it down to keep up with the slower
indexer.
ML> For a reasonable event loop implemented in C, the overhead of
ML> calling select should completely dominate the rest of the
ML> processing (it does so in EV).
true, but bad c (and perl) code is all around us.
>> i will take a gander and see if i can play with it and add stem's loop
>> to it. if you want to work on this with me, i wouldn't mind the help.
ML> Well, can't say thereis much demand for it, but if you cna give me a pointer
ML> on these things in the docs I can probably come up with one before the next
ML> release:
ML> - how can I do one iteration of the event loop? This is important
i don't have an api for oneevent. i still haven't seen a need for
it. hence having stem wrap anyevent may be the better way. the goal for
me is to have stem support more (and faster) event loops. if you are
doing a stem app, you should use the stem event api.
ML> for condvars: the function must not block after one or more events
ML> have been handled.
??
ML> - how do I register a callback to be called when an fh or fd becomes
ML> "readable" or "writable"?
the sessions examples show plenty of that. very similar to event.pm overall.
ML> - how can I register a relative timer?
relative? is that a delay timer (relative to now)? event.pm has hard
(timing from before callback to next trigger) and soft (time from after
callback to next trigger) timers.
ML> - are there any limitations, for example, what happens when I register
ML> two read watchers for one fh (or fd)?
shouldn't be a problem if select handles it. the stem perl event loop is
a wrapper around select.
ML> - does stem provide something to catch signals synchronously?
either use event.pm as the core loop or the pure perl one can do it
under 5.8 and safe signals.
ML> - how about child status changes or exits?
signals are signals. it handles sigchld and the stem::proc module deals
with reaping as needed.
ML> The EV or Event implementation modules should give a good idea of whats
ML> required.
ML> And for the documentation:
ML> - how does stem handle time jumps, if at all?
??
ML> - are it's timers based on absolute or wallclock time or relative/monotonic time?
iirc it checks wallclock (HiRes::time()) after each call to select and
gets its current delta from that. you can't trust a relative clock as it
will skew around.
uri
--
Uri Guttman ------ uri@stemsystems.com -------- http://www.sysarch.com --
----- Perl Code Review , Architecture, Development, Training, Support ------
--------- Free Perl Training --- http://perlhunter.com/college.html ---------
--------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
Thread Previous
|
Thread Next