/src/libeio/eio.pod

  1=head1 NAME
  2
  3libeio - truly asynchronous POSIX I/O
  4
  5=head1 SYNOPSIS
  6
  7  #include <eio.h>
  8
  9=head1 DESCRIPTION
 10
 11The newest version of this document is also available as an html-formatted
 12web page you might find easier to navigate when reading it for the first
 13time: L<http://pod.tst.eu/http://cvs.schmorp.de/libeio/eio.pod>.
 14
 15Note that this library is a by-product of the C<IO::AIO> perl
 16module, and many of the subtler points regarding requets lifetime
 17and so on are only documented in its documentation at the
 18moment: L<http://pod.tst.eu/http://cvs.schmorp.de/IO-AIO/AIO.pm>.
 19
 20=head2 FEATURES
 21
 22This library provides fully asynchronous versions of most POSIX functions
 23dealign with I/O. Unlike most asynchronous libraries, this not only
 24includes C<read> and C<write>, but also C<open>, C<stat>, C<unlink> and
 25similar functions, as well as less rarely ones such as C<mknod>, C<futime>
 26or C<readlink>.
 27
 28It also offers wrappers around C<sendfile> (Solaris, Linux, HP-UX and
 29FreeBSD, with emulation on other platforms) and C<readahead> (Linux, with
 30emulation elsewhere>).
 31
 32The goal is to enable you to write fully non-blocking programs. For
 33example, in a game server, you would not want to freeze for a few seconds
 34just because the server is running a backup and you happen to call
 35C<readdir>.
 36
 37=head2 TIME REPRESENTATION
 38
 39Libeio represents time as a single floating point number, representing the
 40(fractional) number of seconds since the (POSIX) epoch (somewhere near
 41the beginning of 1970, details are complicated, don't ask). This type is
 42called C<eio_tstamp>, but it is guarenteed to be of type C<double> (or
 43better), so you can freely use C<double> yourself.
 44
 45Unlike the name component C<stamp> might indicate, it is also used for
 46time differences throughout libeio.
 47
 48=head2 FORK SUPPORT
 49
 50Calling C<fork ()> is fully supported by this module. It is implemented in these steps:
 51
 52   1. wait till all requests in "execute" state have been handled
 53      (basically requests that are already handed over to the kernel).
 54   2. fork
 55   3. in the parent, continue business as usual, done
 56   4. in the child, destroy all ready and pending requests and free the
 57      memory used by the worker threads. This gives you a fully empty
 58      libeio queue.
 59
 60=head1 INITIALISATION/INTEGRATION
 61
 62Before you can call any eio functions you first have to initialise the
 63library. The library integrates into any event loop, but can also be used
 64without one, including in polling mode.
 65
 66You have to provide the necessary glue yourself, however.
 67
 68=over 4
 69
 70=item int eio_init (void (*want_poll)(void), void (*done_poll)(void))
 71
 72This function initialises the library. On success it returns C<0>, on
 73failure it returns C<-1> and sets C<errno> appropriately.
 74
 75It accepts two function pointers specifying callbacks as argument, both of
 76which can be C<0>, in which case the callback isn't called.
 77
 78=item want_poll callback
 79
 80The C<want_poll> callback is invoked whenever libeio wants attention (i.e.
 81it wants to be polled by calling C<eio_poll>). It is "edge-triggered",
 82that is, it will only be called once when eio wants attention, until all
 83pending requests have been handled.
 84
 85This callback is called while locks are being held, so I<you must
 86not call any libeio functions inside this callback>. That includes
 87C<eio_poll>. What you should do is notify some other thread, or wake up
 88your event loop, and then call C<eio_poll>.
 89
 90=item done_poll callback
 91
 92This callback is invoked when libeio detects that all pending requests
 93have been handled. It is "edge-triggered", that is, it will only be
 94called once after C<want_poll>. To put it differently, C<want_poll> and
 95C<done_poll> are invoked in pairs: after C<want_poll> you have to call
 96C<eio_poll ()> until either C<eio_poll> indicates that everything has been
 97handled or C<done_poll> has been called, which signals the same.
 98
 99Note that C<eio_poll> might return after C<done_poll> and C<want_poll>
100have been called again, so watch out for races in your code.
101
102As with C<want_poll>, this callback is called while lcoks are being held,
103so you I<must not call any libeio functions form within this callback>.
104
105=item int eio_poll ()
106
107This function has to be called whenever there are pending requests that
108need finishing. You usually call this after C<want_poll> has indicated
109that you should do so, but you can also call this function regularly to
110poll for new results.
111
112If any request invocation returns a non-zero value, then C<eio_poll ()>
113immediately returns with that value as return value.
114
115Otherwise, if all requests could be handled, it returns C<0>. If for some
116reason not all requests have been handled, i.e. some are still pending, it
117returns C<-1>.
118
119=back
120
121For libev, you would typically use an C<ev_async> watcher: the
122C<want_poll> callback would invoke C<ev_async_send> to wake up the event
123loop. Inside the callback set for the watcher, one would call C<eio_poll
124()> (followed by C<ev_async_send> again if C<eio_poll> indicates that not
125all requests have been handled yet). The race is taken care of because
126libev resets/rearms the async watcher before calling your callback,
127and therefore, before calling C<eio_poll>. This might result in (some)
128spurious wake-ups, but is generally harmless.
129
130For most other event loops, you would typically use a pipe - the event
131loop should be told to wait for read readyness on the read end. In
132C<want_poll> you would write a single byte, in C<done_poll> you would try
133to read that byte, and in the callback for the read end, you would call
134C<eio_poll>. The race is avoided here because the event loop should invoke
135your callback again and again until the byte has been read (as the pipe
136read callback does not read it, only C<done_poll>).
137
138=head2 CONFIGURATION
139
140The functions in this section can sometimes be useful, but the default
141configuration will do in most case, so you should skip this section on
142first reading.
143
144=over 4
145
146=item eio_set_max_poll_time (eio_tstamp nseconds)
147
148This causes C<eio_poll ()> to return after it has detected that it was
149running for C<nsecond> seconds or longer (this number can be fractional).
150
151This can be used to limit the amount of time spent handling eio requests,
152for example, in interactive programs, you might want to limit this time to
153C<0.01> seconds or so.
154
155Note that:
156
157a) libeio doesn't know how long your request callbacks take, so the time
158spent in C<eio_poll> is up to one callback invocation longer then this
159interval.
160
161b) this is implemented by calling C<gettimeofday> after each request,
162which can be costly.
163
164c) at least one request will be handled.
165
166=item eio_set_max_poll_reqs (unsigned int nreqs)
167
168When C<nreqs> is non-zero, then C<eio_poll> will not handle more than
169C<nreqs> requests per invocation. This is a less costly way to limit the
170amount of work done by C<eio_poll> then setting a time limit.
171
172If you know your callbacks are generally fast, you could use this to
173encourage interactiveness in your programs by setting it to C<10>, C<100>
174or even C<1000>.
175
176=item eio_set_min_parallel (unsigned int nthreads)
177
178Make sure libeio can handle at least this many requests in parallel. It
179might be able handle more.
180
181=item eio_set_max_parallel (unsigned int nthreads)
182
183Set the maximum number of threads that libeio will spawn.
184
185=item eio_set_max_idle (unsigned int nthreads)
186
187Libeio uses threads internally to handle most requests, and will start and stop threads on demand.
188
189This call can be used to limit the number of idle threads (threads without
190work to do): libeio will keep some threads idle in preperation for more
191requests, but never longer than C<nthreads> threads.
192
193In addition to this, libeio will also stop threads when they are idle for
194a few seconds, regardless of this setting.
195
196=item unsigned int eio_nthreads ()
197
198Return the number of worker threads currently running.
199
200=item unsigned int eio_nreqs ()
201
202Return the number of requests currently handled by libeio. This is the
203total number of requests that have been submitted to libeio, but not yet
204destroyed.
205
206=item unsigned int eio_nready ()
207
208Returns the number of ready requests, i.e. requests that have been
209submitted but have not yet entered the execution phase.
210
211=item unsigned int eio_npending ()
212
213Returns the number of pending requests, i.e. requests that have been
214executed and have results, but have not been finished yet by a call to
215C<eio_poll>).
216
217=back
218
219
220=head1 ANATOMY OF AN EIO REQUEST
221
222#TODO
223
224
225=head1 HIGH LEVEL REQUEST API
226
227#TODO
228
229=back
230
231
232=head1 LOW LEVEL REQUEST API
233
234#TODO
235
236=head1 EMBEDDING
237
238Libeio can be embedded directly into programs. This functionality is not
239documented and not (yet) officially supported.
240
241Note that, when including C<libeio.m4>, you are responsible for defining
242the compilation environment (C<_LARGEFILE_SOURCE>, C<_GNU_SOURCE> etc.).
243
244If you need to know how, check the C<IO::AIO> perl module, which does
245exactly that.
246
247
248=head1 COMPILETIME CONFIGURATION
249
250These symbols, if used, must be defined when compiling F<eio.c>.
251
252=over 4
253
254=item EIO_STACKSIZE
255
256This symbol governs the stack size for each eio thread. Libeio itself
257was written to use very little stackspace, but when using C<EIO_CUSTOM>
258requests, you might want to increase this.
259
260If this symbol is undefined (the default) then libeio will use its default
261stack size (C<sizeof (long) * 4096> currently).  If it is defined, but
262C<0>, then the default operating system stack size will be used. In all
263other cases, the value must be an expression that evaluates to the desired
264stack size.
265
266=back
267
268
269=head1 PORTABILITY REQUIREMENTS
270
271In addition to a working ISO-C implementation, libeio relies on a few
272additional extensions:
273
274=over 4
275
276=item POSIX threads
277
278To be portable, this module uses threads, specifically, the POSIX threads
279library must be available (and working, which partially excludes many xBSD
280systems, where C<fork ()> is buggy).
281
282=item POSIX-compatible filesystem API
283
284This is actually a harder portability requirement: The libeio API is quite
285demanding regarding POSIX API calls (symlinks, user/group management
286etc.).
287
288=item C<double> must hold a time value in seconds with enough accuracy
289
290The type C<double> is used to represent timestamps. It is required to
291have at least 51 bits of mantissa (and 9 bits of exponent), which is good
292enough for at least into the year 4000. This requirement is fulfilled by
293implementations implementing IEEE 754 (basically all existing ones).
294
295=back
296
297If you know of other additional requirements drop me a note.
298
299
300=head1 AUTHOR
301
302Marc Lehmann <libeio@schmorp.de>.
303