Module concurrent.futures#
Module concurrent.futures
provides a high-level interface for working with
processes and threads. For both threads and processes the same interface
is used which makes it easy to switch between them.
If you compare this module with threading or multiprocessing, it has fewer
features but with concurrent.futures
it’s easier to work and interface
easier to understand.
Concurrent.futures module allows to solve the problem of starting multiple threads/processes and getting data from them. For this purpose, module uses two classes:
ThreadPoolExecutor
- for threads handlingProcessPoolExecutor
- for process handling
Both classes use the same interface, so it is enough to deal with one and then just switch to other if necessary.
Create an Executor object using ThreadPoolExecutor:
executor = ThreadPoolExecutor(max_workers=5)
After creating an Executor object, it has three methods: shutdown
, map
,
and submit
. Method shutdown
is responsible for the completion of
threads/processes, map
and submit
methods are responsible for
starting functions in different threads/processes.
Note
In fact, map and submit can run not only functions but any callable object. However, only functions will be covered further.
Method shutdown
indicates that Executor object must be finished. However,
if to shutdown
method pass wait=True
(default value), it will not
return the result until all functions running in threads have been completed.
If wait=False
, shutdown
method returns instantly but script itself
will not exit until all functions have been completed.
Generally, shutdown
is not explicitly used because when creating an
Executor object in a context manager, shutdown
is automatically called
at the end of a block with wait=True
.
with ThreadPoolExecutor(max_workers=5) as executor:
...
Since map and submit methods start a function in threads or processes, code must at least have a function that performs one action and must be run in different threads with different arguments of the function.
For example, if you need to ping multiple IP addresses in different threads you need to create a function that pings one IP address and then run this function in different threads for different IP addresses using concurrent.futures.