pooled-io
The motivation for this package was to run computations on multiple cores
that need to write intermediate results to disk.
The functions restrict the number of simultaneously running jobs
to a user given number or to the number of capabilities
the Haskell program was started with,
i.e. the number after the RTS option -N.
There are some flavors of this functionality:
Control.Concurrent.PooledIO.Independent:
run independent actions without results in parallel
Control.Concurrent.PooledIO.Final:
run independent actions with a final result in parallel
Control.Concurrent.PooledIO.InOrder:
run jobs in parallel with data dependencies like make -j n
Additionally there is the module
Control.Concurrent.PooledIO.Sequence
that helps to serialize I/O actions from multiple threads.
It is certainly most useful in connection with
Control.Concurrent.PooledIO.Independent.
Related packages:
lazyio: interleave IO actions in a single thread
async: start threads and wait for their results, forward exceptions,
but do not throttle concurrency with respect to number of available cores
parallel-tasks:
