diff --git a/README.md b/README.md index a9d18d4..cfb6e63 100644 --- a/README.md +++ b/README.md @@ -9,10 +9,10 @@ [![GitHub last commit](https://img.shields.io/github/last-commit/fernandoenzo/parallel-utils)](https://github.com/fernandoenzo/parallel-utils) [![Build Status](https://img.shields.io/travis/com/fernandoenzo/parallel-utils?label=tests)](https://travis-ci.com/fernandoenzo/parallel-utils) -This library implements a class [**Monitor**](https://en.wikipedia.org/wiki/Monitor_(synchronization)), as defined by [**Per +This library implements a [**Monitor**](https://en.wikipedia.org/wiki/Monitor_(synchronization)) class, as defined by [**Per Brinch Hansen**](https://en.wikipedia.org/wiki/Per_Brinch_Hansen) and [**C.A.R. Hoare**](https://en.wikipedia.org/wiki/Tony_Hoare), - for **synchronization and concurrent management of threads and processes in Python**. It also provides **other - functions to ease the creation and collection of results for both threads and processes**. + for **synchronization and concurrent management of threads and processes in Python**. It also provides **additional + functions to facilitate the creation and collection of results for both threads and processes**. ## Table of contents @@ -24,7 +24,7 @@ This library implements a class [**Monitor**](https://en.wikipedia.org/wiki/Moni * [@synchronized](#synchronized) * [Second example](#second-example) * [@synchronized_priority](#synchronized_priority) - * [StaticMonitor class](#staticmonitor-class) + * [StaticMonitor](#staticmonitor) * [Launching threads and processes](#launching-threads-and-processes) * [Contributing](#contributing) * [License](#license) @@ -35,7 +35,7 @@ This library implements a class [**Monitor**](https://en.wikipedia.org/wiki/Moni Use the package manager [**pip**](https://pip.pypa.io/en/stable/) to install **parallel-utils**. ```bash -pip3 install parallel-utils +pip install parallel-utils ``` ## Usage @@ -44,14 +44,14 @@ pip3 install parallel-utils There are two implementations of the `Monitor` class: one is located in the `thread` module and the other in the `process` module of `parallel-utils`. -Although it's safe to always use the `Monitor` class located in the `process` module, even if you're working only with - threads, you will get slightly better performance when using the one located in the `thread` module. Therefore, it is - recommended to use each one for what it is intended for. +Although it's safe to always use the `Monitor` class located in the `process` module, even if you're only working with + threads, you will achieve slightly better performance when using the one located in the `thread` module. Therefore, it is + recommended to use each one for its intended purpose. -From now on, for ease of reading, every time we say _thread_ we will also be including _process_ unless stated otherwise. +For ease of reading, every time we mention a _thread_, we will also be including a _process_ unless stated otherwise.. -Also, from now until the end of this section, when we say _function_, we will not only refer to "whole" functions but we - will also include pieces of code contained within a function. +Also, from now until the end of this section, when we mention a _function_, we will be referring not only to “whole” + functions but also to pieces of code contained within a function. A monitor essentially does two things: 1. It controls the maximum number of threads that can simultaneously access a function. @@ -63,21 +63,21 @@ A monitor essentially does two things: #### First example > 1. It allows controlling the maximum number of threads that can simultaneously access a function. -To achieve this first goal, the Monitor class includes the following couple of functions: +To achieve this first goal, the `Monitor` class includes the following pair of functions: ```python -def lock_code(self, uid: Union[str, int], max_threads: int = 1) +def lock_code(self, uid: str | int, max_threads: int = 1) -def unlock_code(self, uid: Union[str, int]) +def unlock_code(self, uid: str | int) ``` -The first one, `lock_code`, must be called at the beginning of the piece of code for which we want to control the maximum +The first one, `lock_code`, must be called at the beginning of the piece of code for which we want to control the maximum number of threads that can access it simultaneously. The `unlock_code` function sets the limit of the scope of the `lock_code` function. -To do this, both functions must share a same unique identifier (`uid`), that can be either a string or - an integer number. Let's see an example: +To do this, both functions must share the same unique identifier (`uid`), which can be either a string or an integer number. +Let’s see an example: ```python import concurrent.futures @@ -88,7 +88,7 @@ m = Monitor() def print_and_sleep(): print('Hello') - m.lock_code('example', max_threads=1) + m.lock_code(uid='example', max_threads=1) sleep(2) m.unlock_code('example') print('Goodbye') @@ -100,36 +100,52 @@ th3 = create_thread(print_and_sleep) concurrent.futures.wait([th1, th2, th3]) ``` -The example shown above takes 6 seconds to finalize its execution, since we have a `lock_code` that only allows one thread - each time to execute the `sleep` function, and we are launching three threads. +The example shown above takes 6 seconds to complete its execution, since we have a `lock_code` that only allows one thread + at a time to execute the `sleep` function, and we are launching three threads. -If we set the `lock_code` to allow up to three threads at the same time, then the code only needs 2 seconds to finalize - its execution, since all the three threads can make the `sleep` blocking call at the same time. +If we set the `lock_code` to allow up to three threads at the same time, then the code only needs 2 seconds to complete + its execution, since all three threads can make the `sleep` blocking call at the same time. We'll se more about the `create_thread` and `create_process` functions later. -The last line, [`concurrent.futures.wait`](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.wait) - is a blocking call that waits until all the three threads finish running. +The last line, [`concurrent.futures.wait`](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.wait), + is a blocking call that waits until all three threads finish running. +For security and convenience, a context manager, `synchronized`, has been implemented that eliminates the need to explicitly +use `lock_code` and `unlock_code`. This context manager automatically handles the locking and unlocking of code sections, +simplifying the writing of code and improving readability. + +In that case, the function above could be rewritten like this: + +```python +m = Monitor() + +def print_and_sleep(): + print('Hello') + with m.synchronized(uid='example', max_threads=1): + sleep(2) + print('Goodbye') +``` +## #### @synchronized -In the previous example, we just were protecting the piece of code wrapping the `sleep` call. But, what if we want to wrap +In the previous example, we were only protecting the piece of code wrapping the `sleep` call. But, what if we want to wrap the entire function? -Of course we could call `lock_code` on the first line, and `unlock_code` on the last one, and that would work - just fine. Like this: - +Of course, we could use the context manager to wrap the whole code and that would work just fine. Like this: + ```python +m = Monitor() + def print_and_sleep(): - m.lock_code('example', 1) - print('Hello') - sleep(2) - print('Goodbye') - m.unlock_code('example') + with m.synchronized(uid='example', max_threads=1): + print('Hello') + sleep(2) + print('Goodbye') ``` -But, to simplify life for the programmer and for improving readability, there's some syntactic sugar we could use. And this is - where the `@synchronized` decorator comes in and turns the above code into this: +But to simplify life for the programmer and improve readability, there’s some syntactic sugar we could use. And this is +where the `@synchronized` decorator comes in and turns the above code into this: ```python from parallel_utils.thread import synchronized @@ -147,8 +163,8 @@ Let's see the decorator prototype: @synchronized(max_threads: int = 1) ``` -As you can see, the `@synchronized` decorator doesn't need an identifier, and by default only allows one thread to enter the - function at the same time. However, we can override that default behavior with the optional `max_threads` argument. +As you can see, the `@synchronized` decorator doesn’t need an identifier, and by default only allows one thread to enter + the function at the same time. However, we can override that default behavior with the optional `max_threads` argument. If we want, for example, to allow up to two threads to enter the function at the same time, we only need to write: @@ -160,24 +176,24 @@ def print_and_sleep(): print('Goodbye') ``` -Note that **this decorator has its own namespace for uids**, which is completely independent of the namespace of the - `lock_code` and `unlock_code` functions. +Note that **this decorator has its own namespace for uids**, which is completely independent of the namespace of any +`Monitor` class you instantiate. #### Second example ->  2. It organizes a set of functions so that they follow a strict order in their execution, regardless of the thread +> 2. It organizes a set of functions so that they follow a strict order in their execution, regardless of the thread from which they are called. -To achieve the second goal, the Monitor class includes the following couple of functions: +To achieve the second goal, the `Monitor` class includes the following couple of functions: ```python -def lock_priority_code(uid: Union[str, int], order: int = 1, total: int = 1) +def lock_priority_code(uid: str | int, order: int = 1, total: int = 1) -def unlock_code(uid: Union[str, int], order: int) +def unlock_code(uid: str | int, order: int) ``` -Yes, the `unlock_code` function is the same as before. And these two functions work quite similarly to the previous example, - wrapping the code snippet that we want to control and sharing the same `uid` between them. +Yes, the `unlock_code` function is the same as before. And these two functions work quite similarly to the previous +example, wrapping the code snippet that we want to control and sharing the same `uid` between them. The main difference is that, in this case, we have to specify the `order` in which the code snippet will run and the `total` number of functions to sync with the supplied `uid`. @@ -207,7 +223,7 @@ sleep(3) create_process(say_hello, 'Peter') ``` -this example will always print: +This example will always print: ``` Entering goodbye @@ -217,27 +233,47 @@ Hello Peter! Goodbye Peter! ``` -even if you start the `say_goodbye` function long before the `say_hello` function. This is because the snippet in `say_goobye` has - not the first turn, but the second, so it will make a blocking call and wait until `say_hello` calls `unlock_code`. +even if you start the `say_goodbye` function long before the `say_hello` function. This is because the snippet in + `say_goobye` does not have the first turn, but the second, so it will make a blocking call and wait until `say_hello` + calls `unlock_code`. The `total` argument must be supplied **at least once** in any of the calls to `lock_priority_code`. With these two functions, you can sort the execution of as many code snippets as you need. -Note that `lock_code` and `lock_priority_code` share the same namespace for uids. +Note that `lock_code` and `lock_priority_code` share the same namespace for uids, as they are methods of the same +instantiated Monitor, `m`. + +There is also a context manager implemented in this case, similar to before, called `synchronized_priority`, that can +rewrite the above code to look like this: +```python +m = Monitor() + +def say_hello(name): + print('Entering hello') + with m.synchronized_priority('id1', order=1, total=2): + print(f'Hello {name}!') + +def say_goodbye(name): + print('Entering goodbye') + with m.synchronized_priority('id1', order=2): + print(f'Goodbye {name}!') +``` + +## #### @synchronized_priority -Just like before, there is a decorator that we can use to wrap a complete function and set its relative order of execution +Similar to before, there is a decorator that we can use to wrap an entire function and set its relative order of execution with respect to others. This is its prototype: ```python -@synchronized_priority(uid: Union[str, int], order: int = 1, total: int = None) +@synchronized_priority(uid: str | int, order: int = 1, total: int = None) ``` -With it, the last example would look like this: +With it, the previous example would look like this: ```python from parallel_utils.process import create_process, synchronized_priority @@ -257,8 +293,7 @@ sleep(3) create_process(say_hello, 'Peter') ``` -Note that this time we've provided the `total` argument in the second call instead of the first one. Never mind. You - could even supply it in both. +This time, we’ve provided the `total` argument in the second call instead of the first one. You could even supply it in both. The above example will always print: @@ -273,13 +308,14 @@ Goodbye Peter! Note that **this decorator has its own namespace for uids**, which is completely independent of the namespace of the `lock_priority_code` and `unlock_code` functions. -### StaticMonitor class +### StaticMonitor -The `StaticMonitor` class has exactly the same methods as the `Monitor` class. And it also has two implementations: one can - be imported from `parallel_utils.process` and the other can be imported from `parallel_utils.thread`. +For the convenience of programmers, a `Monitor` has already been instantiated and named `StaticMonitor`. Actually, there + are two of them, as usual: one can be imported from `parallel_utils.process` and the other can be imported from + `parallel_utils.thread`. -This class saves you the need to instantiate a `Monitor`, store it in a variable, and then use it. Instead of that, you can - just call its methods like this: +This object saves you the need to instantiate a `Monitor`, store it in a variable, and then use it. Instead, you can just + call its methods like this: ```python from parallel_utils.process import StaticMonitor @@ -291,12 +327,23 @@ def say_hello(name): StaticMonitor.unlock_code('id1') ``` -Note that this class, since it is static, has a unique namespace for uids that is shared among all calls to its methods. +or better: + +```python +from parallel_utils.process import StaticMonitor + +def say_hello(name): + print('Entering hello') + with StaticMonitor.synchronized_priority('id1', order=1, total=2): + print(f'Hello {name}!') +``` + +Note that this object has a unique namespace for uids that is shared among all calls to its methods. ### Launching threads and processes -This library includes two very useful functions to quickly start processes and threads, and retrieve their results, which we - have already seen in our examples: +This library includes two very useful functions to quickly start processes and threads, and retrieve their results, which + we have already seen in our examples: ```python def create_thread(func: Callable, *args: Any, **kwargs: Any) -> Future @@ -304,10 +351,10 @@ def create_thread(func: Callable, *args: Any, **kwargs: Any) -> Future def create_process(func: Callable, *args: Any, **kwargs: Any) -> Future ``` -Like the rest of classes and objects in this library, they are located in `parallel_utils.thread` and +Like the rest of the classes and objects in this library, they are located in `parallel_utils.thread` and `parallel_utils.process` respectively. -Their first argument is a Callable that, in turn, is called with `*args` and `**kwargs`. +Their first argument is a `Callable` that, in turn, is called with `*args` and `**kwargs`. They both return a [`Future`](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future) object, which encapsulates the asynchronous execution of a callable.