Labs / Python parallel computing

A Quick Guide to Python Parallel Computing: 03 Python Multithreaded Programming - Multithread Synchronization (Part 1) — Based on Mutex Locks, Recursion Locks and Semaphores

Author: Li Zisheng · Source: FreakStudio open documentation · License: CC BY 4.0

Contents
A Quick Guide to Python Parallel Computing: 03 Python Multithreaded Programming - Multithread Synchronization (Part 1) — Based on Mutex Locks, Recursion Locks and Semaphores

Abstract:

In Python multi-threaded programming, thread synchronization is a key technology to ensure that multiple threads access shared resources safely. This article introduces the concepts and applications of mutex locks (Lock), recursion locks (RLock) and semaphores (Semaphore). Mutex locks are used to prevent multiple threads from modifying data simultaneously, recursion locks are suitable for nested locking scenarios, while semaphores limit the number of threads that can access a resource at the same time.

347622978

For more information about us, you can refer to the cloud document: Freak Embedded Studio Cloud Docs, or visit our wiki: https://github.com/leezisheng/Doc/wik

FreakStudio's Blog

Previous Recommendations:

You who are learning embedded systems, don’t you even know Object Oriented yet? ?!

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 00 Introduction to Object Oriented Design Methodology

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 01 Basic Concepts of Object Oriented Programming

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 02 Python Implementation of Classes and Objects - Creating Classes with Python

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 03 Python Implementation of Classes and Objects – Adding Attributes to Custom Classes

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 04 Python Implementation of Classes and Objects - Adding Methods to Custom Classes

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 05 Python Implementation of Classes and Objects - PyCharm Code Tab

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 06 Python Implementation of Classes and Objects - Data Encapsulation of Custom Classes

The most beginner-friendly Object Oriented programming tutorial across the entire network: 07 Python implementation of classes and objects - type annotations

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 08 Python Implementation of Classes and Objects - @property Decorator

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 09 Python Implementation of Classes and Objects - Relationships Between Classes

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 10 Python Implementation of Classes and Objects - Class Inheritance and Liskov Substitution Principle

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 11 Python Implementation of Classes and Objects - Subclass Calling Parent Class Methods

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 12 Python Implementation of Classes and Objects - Using Python's logging Module to Output Program Runtime Logs

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 13 Python Implementation of Classes and Objects – Installation and Usage of Sourcetrail, a Visual Code Reading Tool

The Most Beginner-Friendly Object Oriented Programming Course on the Internet: The Most Beginner-Friendly Object Oriented Programming Course on the Internet: 14 Python Implementation of Classes and Objects - Static Methods and Class Methods of Classes

The most beginner-friendly Object Oriented programming tutorial across the entire network: 15 Python implementation of classes and objects - the __slots__ magic method

The Most Beginner-Friendly Object Oriented Programming Tutorial Across the Internet: 16 Python Implementation of Classes and Objects - Polymorphism, Method Overriding and Open Closed Principle

The most beginner-friendly Object Oriented programming tutorial across the network: 17 Python implementation of classes and objects - duck typing and "file-like object"

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 18 Python Implementation of Classes and Objects - Multiple Inheritance and Drawing Curve Graphs of Serial Port Data with PyQtGraph

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 19 Python Implementation of Classes and Objects – Using PyCharm to Automatically Generate File Comments and function Comments

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 20 Python Implementation of Classes and Objects – Implementation of Composition Relationship and CSV File Saving

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 21 Python Implementation of Classes and Objects - Multi-File Organization: Module and Package

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 22 Python Implementation of Classes and Objects - Exceptions and Syntax Errors

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 23 Python Implementation of Classes and Objects - Throwing Exceptions

The Most Beginner-Friendly Object Oriented Programming Tutorial Across the Network: 24 Python Implementation of Classes and Objects - Exception Capture and Handling

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 25 Python Implementation of Classes and Objects - Python Determines Input Data Types

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 26 Python Implementation of Classes and Objects - Context Managers and the with Statement

The Most Beginner-Friendly Object Oriented Programming Tutorial Across the Network: 27 Python Implementation of Classes and Objects - Implementation of Exception Hierarchy and Custom Exception Classes in Python

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 28 Python Implementation of Classes and Objects - A Comprehensive Summary of Python Programming Principles, Philosophy and Specifications

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 29 Python Implementation of Classes and Objects - Assertions, Defensive Programming and the Usage of the help function

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 30 Python's Built-in Data Types - The object Root Class

The most beginner-friendly Object Oriented programming tutorial across the entire network: 31 Built-in data types in Python - Object and Type

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 32 Python's Built-in Data Types - Class and Instance

The most beginner-friendly Object Oriented programming tutorial on the entire network: 33 Built-in data types in Python – the relationship between objects and types

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 34 Python's Built-in Data Types - Commonly Used Composite Data Types in Python: Tuples and Named Tuples

The most beginner-friendly Object Oriented programming tutorial on the entire network: 35 Python's built-in data types - docstrings and the __doc__ attribute

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 36 Python's Built-in Data Type - Dictionary

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 37 Common Composite Data Types in Python - Lists and List Comprehensions

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 38 Common Composite Data Types in Python – Implementing stack, Queue and Deque with Lists

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 39 Common Composite Data Types in Python - Sets

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 40 Common Composite Data Types in Python - Enumerations and Usage of the enum Module

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 41 Common Composite Data Types in Python - Queues (FIFO, LIFO, Priority Queue, Deque and Circular Queue)

The Most Beginner-Friendly Object Oriented Programming Tutorial Across the Network: 42 Commonly Used Composite Data Types in Python - Collections Container Data Types

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 43 Common Composite Data Types in Python - Extending Built-in Data Types

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 44 Python Built-in functions and Magic Methods – Overriding Magic Methods of Built-in Types

The Most Beginner-Friendly Object Oriented Programming Course on the Whole Network: 45 Implementing Common Data Structures in Python – Linked List, Tree, Hash Table, Graph and Heap

The most beginner-friendly Object Oriented programming tutorial across the network: 46 Python function methods and interfaces - functions and event-driven frameworks

The most beginner-friendly Object Oriented programming tutorial across the network: 47 Python function methods and interfaces - Callback function

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 48 Python function Methods and Interfaces - Positional Arguments, Default Arguments, Variable Arguments and Keyword Arguments

The Most Beginner-Friendly Object Oriented Programming Tutorial Across the Network: 49 Python function Methods and Interfaces – The Difference Between function and Methods, and Lambda Anonymous Functions

The most beginner-friendly Object Oriented programming tutorial on the entire network: 50 Python function methods and interfaces - Interfaces and Abstract Base Classes

The most beginner-friendly Object Oriented programming tutorial on the entire network: 51 Python function methods and interfaces - Implementing interfaces with Zope

The most beginner-friendly Object Oriented programming tutorial across the network: 52 Python function methods and interfaces - Protocol and interfaces

The most beginner-friendly Object Oriented programming tutorial across the entire network: 53 Python Strings and Serialization - Strings and Character Encodings

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 54 Python Strings and Serialization - String Formatting and the format Method

The most beginner-friendly Object Oriented programming tutorial on the entire network: 55 Python Strings and Serialization - Byte Sequence Types and Mutable Byte Strings

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 56 Python Strings & Serialization - Regular Expression and re Module Application

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 57 Python Strings and Serialization - Serialization and Deserialization

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 58 Python Strings and Serialization – Definition and Implementation of Serialized Web Objects

The Most Beginner-Friendly Object Oriented Programming Tutorial on the Whole Network: 59 Python Parallelism and Concurrency - Parallelism & Concurrency and Threads & Processes

Master Python Parallel Computing in One Article: 00 Basic Concepts of Parallel Computing

A Quick Guide to Python Parallel Computing: 01 Python Multithreaded Programming – Basic Concepts, Switching Process, GIL Lock Mechanism, and Producer-Consumer Model

A Quick Guide to Python Parallel Computing: 02 Python Multithreaded Programming - threading Module, Thread Creation & Query, and Daemon Threads

More exciting content is available here:

Speed Up Your Python: A Quick Guide to Python Parallel Computing

Understand the Debugging Principle of CM3 Microcontroller in One Article

After half a month of dedicated work, the ultimate summary of embedded technology stack is now released

The "Martial Arts Manual" for Computer Science Competitions

An open-source project collection for MicroPython: awesome-micropython, which covers MicroPython tool libraries in various aspects

Avnet ZUBoard 1CG Development Board — A New Choice for Deep learning

Engineers should not blindly worship open-source code; they also need to focus on mastering fundamental skills.

What? A motor drive module with customizable color schemes? !

What? Three brand-new expansion boards have just been released for the XIAO main controller!

A Step-by-Step Guide to Publishing Your Third-Party Library for Arduino

A 10,000-Word Comprehensive Guide: Step-by-Step Instructions to Publish Third-Party Libraries for MicroPython/Python

Document Acquisition:

You can download the document via the following link:

https://github.com/leezisheng/Doc

This document is a learning guide on parallel computing and Python concurrent programming, which covers the core knowledge points including the basic concepts of parallel computing, Python multi-threaded programming, multi-process programming and coroutine programming:

10 1.2

Body Text

1. Basic Concepts of Thread Synchronization

In Python multithreaded programming, thread synchronization is a mechanism that ensures multiple threads can safely access shared resources. In a multithreaded environment, if multiple threads access and modify shared resources simultaneously, issues such as race conditions and data inconsistency may occur. For example:

import threading

# Shared variable
counter = 0

def increment():
    global counter
    for _ in range(1000000):
        counter += 1

# Create two threads
t1 = threading.Thread(target=increment)
t2 = threading.Thread(target=increment)

# Start the threads
t1.start()
t2.start()

# Wait for the threads to finish
t1.join()
t2.join()

# Expected to be 2000000, but may be less than this value
print("Final Counter:", counter)  

Since counter += 1 is not an atomic operation, but a three-step process of read -> compute -> write, two threads may read counter at the same time, causing partial data loss when writing. The final result may be less than 2000000, which is the so-called Race Condition.

Let's take another practical example: in the code below, we spawn 200 threads, and all of these 200 threads will access counter, this shared resource, and perform an operation on it (counter += 1). Let's take a look at the running result:

import threading
import time
 
counter = 0
 
class MyThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
 
    def run(self):
        global counter
        time.sleep(1);
        counter += 1
        print("I am %s, set counter:%s" % (self.name, counter))
 
if __name__ == "__main__":
    for i in range(0, 200):
        my_thread = MyThread()
        my_thread.start()
200 Thread-184 185 Thread-200 0

From this, we have already seen the preemption of this global resource (counter). The root cause of the problem is that there is no proper control over multiple threads accessing the same resource, which leads to data corruption and makes the execution results of threads unpredictable. This phenomenon is referred to as "thread-unsafe", and we must avoid such situations during the development process.

2. Thread Data Synchronization Based on Mutex Locks

The simplest solution to "thread-unsafe" issues is to use locks. The operation of a lock is straightforward: when a thread needs to access a segment of shared memory, it must first acquire the lock before gaining access. After the thread finishes using this portion of shared resources, it must release the lock, so that other threads can then acquire the lock and access the same resources.

In Python, we use the Lock class provided by the threading module to implement the mutex mechanism. To modify the above program, we need to add a lock variable mutex = threading.Lock (), then before competing for resources, we will first acquire this lock mutex. acquire (), and after finishing using the resource, we will release this lock mutex. release ().

The code is as follows:

import threading
import time

counter = 0
mutex = threading.Lock()

class MyThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        global counter, mutex
        time.sleep(1);
        if mutex.acquire():
            counter += 1
            print("I am %s, set counter:%s" % (self.name, counter))
            mutex.release()

if __name__ == "__main__":
    for i in range(0, 100):
        my_thread = MyThread()
        my_thread.start()

The output shown in the figure below indicates that the counter variable keeps incrementing, and the preemption issue of the global resource (counter) has been resolved. The main process is as follows: when a thread calls the Lock object's acquire () method to acquire the lock, the lock enters a "locked" state. Since only one thread (Thread 1) can acquire the lock at a time, if another thread (Thread 2) attempts to acquire this lock at this point, that thread (Thread 2) will enter a "block" synchronous blocking state.Until the thread that holds the lock, Thread 1, calls the lock's release () method to release the lock, the lock enters the "unlocked" state. The thread scheduler selects one of the threads in the synchronous blocking state to acquire the lock, and transitions this thread to the running state.

Thread-83 Thread-100

However, in practical use, we find that this method frequently gives rise to a problematic deadlock situation. A deadlock occurs when different threads attempt to acquire a lock, at which point the program can no longer proceed with execution, as each thread holds the lock that the other requires.

1 2

To simplify the problem, let us assume that there are two concurrent threads (Thread A and Thread B) that require Resource 1 and Resource 2. Suppose Thread A needs Resource 1, and Thread B needs Resource 2. In this scenario, both threads are using their respective locks, and there is no conflict so far. Now suppose that before either side releases its lock, Thread A requests the lock for Resource 2, and Thread B requests the lock for Resource 1; neither thread will proceed without the required resource.Given that the locks of both resources are currently occupied, and each side remains in a waiting state without releasing its own lock until the other side's lock is freed, this is a typical scenario of deadlock.

Let's look at the following code:

import threading

counterA = 0
counterB = 0

mutexA = threading.Lock()
mutexB = threading.Lock()

class MyThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        self.fun1()
        self.fun2()

    def fun1(self):
        global mutexA, mutexB
        if mutexA.acquire():
            print("I am %s , get res: %s" % (self.name, "ResA"))

            if mutexB.acquire():
                print("I am %s , get res: %s" % (self.name, "ResB"))
                mutexB.release()

        mutexA.release()

    def fun2(self):
        global mutexA, mutexB
        if mutexB.acquire():
            print("I am %s , get res: %s" % (self.name, "ResB"))

            if mutexA.acquire():
                print("I am %s , get res: %s" % (self.name, "ResA"))
                mutexA.release()

        mutexB.release()

if __name__ == "__main__":
    for i in range(0, 100):
        my_thread = MyThread()
        my_thread.start()

The code demonstrates that two function functions of a thread acquire another competing resource after having acquired one competing resource respectively. Let's look at the running result:

Thread-31

As you can see, the program has been suspended there, and this phenomenon is what we call a "deadlock". The main method to avoid deadlocks is to allocate resources correctly and in an orderly manner. The most representative algorithm in deadlock avoidance algorithms is the Banker's Algorithm proposed by Dijkstra E.W in 1968.

3. Thread Data Synchronization Based on Recursion Lock

Consider this scenario: what should a thread do when encountering lock nesting? Here, nesting refers to the situation where a thread, while already holding access to a critical resource, needs to acquire that same resource again. The corresponding code is as follows:

import threading
import time

counter = 0
mutex = threading.Lock()

class MyThread(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)

    def run(self):
        global counter, mutex
        time.sleep(1);
        if mutex.acquire():
            counter += 1
            print("I am %s, set counter:%s" % (self.name, counter))
            if mutex.acquire():
                counter += 1
                print("I am %s, set counter:%s" % (self.name, counter))
                mutex.release()
            mutex.release()
if __name__ == "__main__":
    for i in range(0, 200):
        my_thread = MyThread()
        my_thread.start()

The code execution in this scenario is as follows: as can be seen, after a thread acquires the mutex once, it needs to use the critical variable counter again, so it attempts to acquire the mutex once more:

Demo/00 1

then it directly hangs, and this situation forms the simplest deadlock.

Is there a situation where a thread can access a competing resource using mutual exclusion and retrieve it again? In Python, to support multiple requests for the same resource in the same thread, Python provides a "reentrant lock": threading. RLock . This RLock maintains a Lock and a counter variable internally. Counter records the number of acquire , so that the resource can be required multiple times.Only when all the acquire operations of a thread have been release d can other threads acquire the resource. If we use RLock instead of Lock in the above example, no deadlock will occur:

The code only needs to implement the following:

mutex = threading.Lock()

Replace with:

mutex = threading.RLock()
Thread-192 384

Compared with Lock, this lock has three characteristics:

Nested locks also have drawbacks, as they create complications for our lock detection mechanisms.

4. Thread Data Synchronization Based on Semaphores

Semaphores were invented by E. Dijkstra and first applied in operating systems. As an abstract data type managed by the operating system, semaphores are used to synchronize the access to shared resources among multiple threads. In essence, a semaphore is an internal data that indicates how many concurrent reads are allowed for the current shared resource.

Similarly, in the threading module, there are two functions for semaphore operations: acquire () and release (), which are explained as follows:

(1) Whenever a thread intends to read the shared resource associated with the semaphore, it must call acquire (). This operation decrements the internal variable of the semaphore; if the value of this variable is non-negative, the permission to allocate the resource is granted. If the value is negative, the thread is suspended until another thread releases the resource;

(2) When a thread no longer needs the shared resource, it must release the resource via release (). This increments the internal variable of the semaphore, and the thread at the head of the semaphore's waiting queue will obtain access to the shared resource.

A Quick Guide to Python Parallel Computing: 03 Python Multithreaded Programming - Multithread Synchronization (Part 1) — Based on Mutex Locks, Recursion Locks and Semaphores

Generally speaking, to acquire a shared resource, a thread needs to perform the following operations:

  1. Test the semaphore that controls this resource.
  2. If the value of this semaphore is positive, the resource is allowed to be used, and the thread decrements the semaphore by 1.
  3. If the value of this semaphore is 0, the resource is currently unavailable, and the thread enters a sleeping state; it will not be woken up and transferred to step 1 until the semaphore value becomes greater than 0.
  4. When a thread no longer uses the resource controlled by a semaphore, the value of the semaphore is incremented by 1. If there is a thread sleeping and waiting for this semaphore at this time, wake up this thread.

In the following code, we use the producer-consumer model to demonstrate synchronization via semaphores. We have two threads, producer () and consumer (), which share a common resource item. producer () is tasked with producing item, while consumer () is tasked with consuming item. When the item has not yet been produced, consumer () waits continuously; once the item is produced, the producer () thread notifies the consumer that the resource is available.

import threading
import time
import random

# By initializing the semaphore to 0, we get a so-called semaphore event
# The optional Semaphore argument gives the initial value of the internal variable, default 1.
# If the given value is less than 0, ValueError is raised.
semaphore = threading.Semaphore(0)

def consumer():
        print("consumer is waiting.")

        if semaphore.acquire():
                # If the consumer acquires the semaphore, i.e. the semaphore is greater than 0
                print("Consumer notify : consumed item number %s " % item)
        # If the consumer does not acquire the semaphore, i.e. the semaphore equals 0, the consumer thread blocks

def producer():
        global item
        time.sleep(1)

        item = random.randint(0, 1000)
        print("producer notify : produced item number %s" % item)
        #  Release the semaphore to notify the consumer thread
        semaphore.release()

if __name__ == '__main__':
        for i in range (0,5):
                t1 = threading.Thread(target=producer)
                t2 = threading.Thread(target=consumer)
                t1.start()
                t2.start()
                t1.join()
                t2.join()
        print("program terminated")

Check the code running result:

80 0

A special use of semaphores is the mutex. A mutex is a semaphore initialized to 1, which enables mutually exclusive access to data and resources. Semaphores are still widely used in programming languages that support multithreading, yet this may lead to deadlock situations.

threading.BoundedSemaphore is used to implement a bounded semaphore. A bounded semaphore checks to ensure that its current value never exceeds its initial value; if it does, a ValueError exception will be raised. In most scenarios, semaphores are used to protect a limited number of resources. If a semaphore is released too many times, it indicates a bug. If not specified, the value parameter defaults to 1.