Labs / Python parallel computing

A Quick Guide to Python Parallel Computing: A Tutorial Summary

Author: Li Zisheng · Source: FreakStudio open documentation · License: CC BY 4.0

Contents
A Quick Guide to Python Parallel Computing: A Tutorial Summary

Abstract:

This tutorial is a systematic practical guide to Python parallel computing. Starting from the basic concepts of parallel computing, it progressively and in-depth elaborates on the three core paradigms for implementing concurrency in Python: multithreading, multiprocessing, and asynchronous programming. It not only conducts a detailed analysis of the GIL mechanism in multithreading, various synchronization primitives (locks, semaphores, condition variables, etc.) and communication methods (queues), but also clearly compares the advantages of multiprocessing in breaking through GIL limitations and realizing true parallel computing, covering multiple solutions for inter-process data sharing and communication (shared memory, managers, pipes).Finally, the tutorial introduces a modern asynchronous programming model, explaining how event loops and coroutines efficiently handle highly I/O-intensive tasks. With a well-structured, progressive design from basics to advanced topics across the entire series, it provides developers with a complete knowledge system spanning theory to practice, making it an excellent tutorial for quickly mastering and applying Python to improve program performance.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

For more information about us, you can refer to the cloud document: Freak Embedded Studio Cloud Docs, or visit our wiki: https://github.com/leezisheng/Doc/wik

FreakStudio's Blog

Previous Recommendations:

Master Python Parallel Computing in One Article: 00 Basic Concepts of Parallel Computing

A Quick Guide to Python Parallel Computing: 01 Python Multithreaded Programming – Basic Concepts, Switching Process, GIL Lock Mechanism, and Producer-Consumer Model

A Quick Guide to Python Parallel Computing: 02 Python Multithreaded Programming — the threading Module, Thread Creation & Query, and Daemon Threads

A Quick Guide to Python Parallel Computing: 03 Python Multithreaded Programming - Multithread Synchronization (Part 1) — Based on Mutex Locks, Recursion Locks and Semaphores

A Quick Guide to Python Parallel Computing: 04 Python Multithreaded Programming – Multithread Synchronization (Part 2) — Based on Condition Variables, Events and Barriers

A Quick Guide to Python Parallel Computing: 05 Python Multithreaded Programming – Scheduled Thread Execution

A Quick Guide to Python Parallel Computing: 06 Python Multithreaded Programming – Communication Based on Queues

A Quick Guide to Python Parallel Computing: 07 Python Multithreaded Programming – Using Thread Pools and Performance Evaluation of Multithreading

A Quick Guide to Python Parallel Computing: 08 Python Multiprocessing Programming – Process Creation & Naming, ID Retrieval, Daemon Process Setup and Process Termination

A Quick Guide to Python Parallel Computing: 09 Python Multiprocessing Programming – Data Synchronization Between Processes: Based on Mutex Locks, Recursion Locks, Semaphores, Condition Variables, Events and Barriers

Master Python Parallel Computing in One Read: 10 Python Multiprocessing Programming - Data Sharing Between Processes - Based on Shared Memory and Data Managers

A Quick Guide to Python Parallel Computing: 11 Python Multiprocessing Programming — Secure Data Transmission Between Processes Based on Queues and Pipes

A Quick Guide to Python Parallel Computing: 12 Python Multiprocessing Programming - Process Pool

A Quick Guide to Python Parallel Computing: 13 Python Asynchronous Programming – Basic Concepts, Event Loop and Callback Mechanism

A Quick Guide to Python Parallel Computing: 14 Python Asynchronous Programming – Coroutine Management and Scheduling

For more exciting content, you can check out:

C Language 1.5 Programming Practice: Pure C Modularization × Inheritance × Polymorphism Framework

Speed Up Your Python: A Quick Guide to Python Parallel Computing

Mastering the Debugging Principles of CM3 Microcontrollers in One Article

After half a month of dedicated grinding, the ultimate roundup of embedded technology stack is now ready

The "Secret Martial Arts Manual" for Computer Science Competitions

An open-source project collection for MicroPython: awesome-micropython, which covers MicroPython tool libraries in various aspects

Avnet ZUBoard 1CG Development Board — A New Choice for Deep learning

Engineers should not blindly worship open-source code, but also pay attention to mastering basic skills

What? A motor drive module with customizable color schemes? !

What? Three new expansion boards just released for the XIAO main controller!

A Step-by-Step Guide to Publishing Third-Party Libraries for Arduino

A 10,000-Word Detailed Guide: Step-by-Step Tutorial on Publishing Third-Party Libraries for MicroPython/Python

【Electronic DIY Magic Tool】Compatible with All Kinds of 5-Wire Stepper Motors! Supports seeed XIAO Series Development Boards

Master the Electronic Design Contest in One Read: The Must-Know Winning Guide for Electronics Enthusiasts

Master the Optoelectronic Design Competition in One Read: A Must-See for Electronics Enthusiasts!

"Brainless" Operation Guide for Engineering Competitions: A Step-by-Step Roadmap from Knowledge Learning, hardware Selection, Code Debugging to Report Writing

How to Win the National Award of the Datang Cup with Zero Foundation? All the Senior's Competition Preparation Tips Are Here

【Extensively Compiled In-Depth Guide】College Student Competition Crash Course: Topic Selection × Team Formation × Roadshow, Get Competition-Ready in 48 Hours

Chaos in Contemporary College Students' Competitions: A Survival Guide from "Involution" to "Heirloom Projects"

Charging over 6,000 yuan for a single conference shoot? Here are all the shooting tips and steps

A female college student's on-site research on losing 5,000 yuan from a street stall: Where in Chengdu has the largest crowd and is easiest to make money? I'll tell you!

Making Money with Polaroid Cameras at Chengdu’s Popular Check-in Spots: A Super Practical Street-Vendor-Level Side Hustle Guide for Tourists

Impressions of Chengdu: An Electronic Girl Created a Touch of Romance by Herself

General Relays vs. Latching Relays vs. MOSFETs: A Complete Analysis of Working Principles and Circuit Design

Goodbye TP4056! Domestic SY3501D single chip integrates charging, discharging and boosting functions with only 7 components! Open source PCB files attached

POB: Programming Oriented to Boss — A Novel Reality-Driven Programming Paradigm

Document Acquisition:

You can download the document via the following link:

https://github.com/leezisheng/Doc

This document is a learning guide on parallel computing and Python concurrent programming, which covers the basic concepts of parallel computing, as well as the core knowledge points of Python multi-threaded programming, multi-process programming and coroutine programming:

A Quick Guide to Python Parallel Computing: A Tutorial Summary

Main Text

00 Basic Concepts of Parallel Computing

This paper systematically introduces the basic knowledge of parallel computing. Firstly, it compares the differences between serial computing and parallel computing, and illustrates the basic characteristics of parallel computing. Then it presents the key concepts in parallel computing, including task parallelism, Data Parallelism, degree of parallelism, parallel algorithms and parallel programming. After that, it expounds the main contents of learning parallel computing, such as parallel algorithms and programming models, Performance optimization methods and common challenges.This paper also illustrates several typical parallel programming models (shared memory, message passing, Data Parallelism, and task parallelism), and discusses the performance evaluation methods for parallel programs, including parallel efficiency, speedup ratio, and Amdahl's Law. Finally, this paper presents the environment configuration required for learning parallel programming, and provides suggestions for using Python and related tools.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

01 Python Multithreaded Programming – A Quick Guide to Basic Concepts, Switching Process, GIL Lock Mechanism, Producer-Consumer Model, and Python Parallel Computing

This paper mainly introduces the basic concepts and applications of multithreading in Python. It first explains the advantages of multithreading, such as improving program operation efficiency, enhancing user experience and making better use of resources, and points out the differences between threads and processes. Then, this paper elaborates on the definition of threads in Python, the different states of threads and their switching processes, to help understand the behaviors of threads in the stages of creation, ready, running, blocking and termination.Meanwhile, this paper also elaborates on the Global Interpreter Lock (GIL) mechanism of Python and its impact on multi-threaded performance, and analyzes the different behaviors of multi-threading in I/O-intensive and CPU-intensive tasks. Finally, this paper introduces the classic producer-consumer model, and expounds on elements such as buffer, producer, consumer, mutex lock and condition variable, which lays a foundation for the subsequent in-depth study of Python multi-threaded programming.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

02 Multithreaded Programming in Python - threading Module, Thread Creation & Query, and Daemon Threads

This document introduces the threading module in Python 3 and its application in multi-threaded programming. It first explains the function and common components of the threading module, such as Thread, Lock, RLock, Condition, Event, Semaphore, etc. Then it focuses on two thread creation methods: calling the constructor of the Thread class, and inheriting the Thread class and overriding the run () method.Subsequently, this paper demonstrates common operations of threads, including obtaining thread names, determining whether a thread is alive, and using join () to control the execution order of threads. Finally, it also introduces the usage of daemon threads and explains the relationship between daemon threads and the main thread.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

03 Python Multithreaded Programming - Multithread Synchronization (Part 1): Based on Mutex Locks, Recursion Locks and Semaphores

This article mainly introduces the thread synchronization mechanism in Python multithreading. First, it explains the race conditions and thread-unsafe problems that may occur when multiple threads access shared resources, and intuitively demonstrates the situation of data inconsistency through examples. Then, it introduces a variety of common synchronization methods: using mutual exclusion lock (Lock) to ensure the mutual exclusion of resource access, and points out the risk of deadlock that it may cause; using recursion lock (RLock) to solve the nested deadlock problem caused by multiple locking in the same thread;and the use of semaphores (Semaphore / BoundedSemaphore) to coordinate multi-threaded access to limited resources, with explanations provided via the producer-consumer model. Together, these methods form a critical foundation for implementing thread safety in Python multi-threaded programming.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

04 Python Multithreaded Programming - Multithread Synchronization (Part 2): Based on Condition Variables, Events and Barriers

This article introduces advanced thread synchronization mechanisms in Python, namely Condition, Event and Barrier. Condition variables are commonly used to solve complex thread collaboration problems, such as the producer-consumer model, and implement precise conditional wake-up via wait () and notify (); the event mechanism controls the timing of thread execution through set () and wait (), which is suitable for scenarios where multiple threads are woken up uniformly after resources are ready;Barriers are used for mutual waiting among multiple threads to ensure that all threads reach the synchronization point before proceeding with execution. The combination of these three can effectively address more complex synchronization requirements in multi-threaded programming.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

05 Python Multithreaded Programming - Scheduled Execution of Threads

This article introduces multiple methods for implementing scheduled tasks in Python. First, threading.Timer is used to implement simple delay or periodic tasks; then the standard library sched is utilized for one-off task scheduling; next, threading.Event is adopted to implement periodic tasks that can be terminated at any time; finally, the powerful third-party library APScheduler is presented, which supports multiple scheduling modes including interval execution, specific-time execution and Cron expression scheduling, and is suitable for complex scheduled requirements.Through comparison, it can be found that different methods are applicable to different scenarios, covering everything from lightweight to enterprise-level applications.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

06 Multithreaded Programming in Python - Communication Based on Queues

This article introduces the basic concepts and common implementation methods of queues in Python. First, it demonstrates the producer-consumer model via queue.Queue, then discusses solutions for the "slow production, fast consumption" scenario; subsequently, it illustrates the usage of the last-in-first-out queue LifoQueue; finally, it presents the priority-based PriorityQueue, which retrieves elements in order of priority and is suitable for scenarios such as task scheduling. Through these examples, readers can fully grasp the diverse applications of queues in Python multi-threaded programming.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

07 Python Multithreaded Programming – Usage of Thread Pool and Performance Evaluation of Multithreading

This paper first introduces the basic concept of thread pool and its advantages in improving performance and controlling the number of concurrencies, and focuses on explaining the common methods of ThreadPoolExecutor and the usage of Future objects, including typical usages such as task submission, result acquisition, callback function and map method, demonstrating the convenience of thread pool in simplifying concurrent programming.Subsequently, the article also evaluates the multi-threaded performance using the tool VizTracer, illustrates the impact of the Global Interpreter Lock (GIL) on the parallel execution of Python multi-threading, and demonstrates how to use visual analysis tools to conduct an in-depth analysis of the execution process of multi-threaded programs.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

08 Python Multiprocessing Programming - Process Creation and Naming, ID Acquisition, Daemon Process Creation and Process Termination

This article introduces the multiprocessing module in Python and its critical role in making full use of multi-core CPU resources. It first explains how to create and start child processes via the Process class, covering two approaches: directly invoking the constructor and inheriting from the Process class; it then demonstrates methods for process naming and PID acquisition to facilitate more intuitive debugging and management. After that, the article also elaborates on the concept and implementation of daemon processes, clarifying the feature that such background processes exit along with the main process;Finally, it explains process termination and lifecycle management, including the usage of terminate (), is_alive () and exitcode. Together, these contents form the core foundation of Python multiprocessing programming.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

09 Python Multiprocessing Programming - Data Synchronization Between Processes: Based on Mutex Locks, Recursion Locks, Semaphores, Condition Variables, Events and Barriers

This paper introduces the common mechanisms for process synchronization in the multiprocessing module of Python. Since data between processes is independent by default while they share the file system, competition and disorder will occur without proper control, thus synchronization primitives are required for coordination.This article sequentially explains the usage methods and application scenarios of mutual exclusion locks (Lock), recursion locks (RLock), semaphores (Semaphore), condition variables (Condition), events (Event), and barriers (Barrier), and demonstrates their synchronization effects in multi-process scenarios with sample code. These synchronization tools can effectively avoid resource contention and ensure the correctness of inter-process collaboration.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

10 Python Multiprocessing Programming - Data Sharing Between Processes: Based on Shared Memory and Data Manager

This paper introduces the implementation methods of inter-process data sharing in the Python multiprocessing module. It first explains the usage of shared memory, including the two basic data type sharing approaches of Value and Array, as well as custom structures defined based on ctypes to achieve efficient data exchange among multiple processes. Shared memory boasts the advantages of high read-write efficiency and no need for data copying, but it needs to be used in conjunction with a synchronization mechanism to avoid data conflicts.Subsequently, the article introduces the data manager (Manager), which maintains shared objects through service processes, supports multiple common data structures such as dict, list, and allows different processes to collaboratively modify the same data. Compared with shared memory, Manager is more flexible to use and is suitable for sharing complex objects.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

11 Python Multiprocessing Programming - Secure Data Transmission Between Processes - Based on Queues and Pipes

This article introduces two main approaches to Inter-Process Communication (IPC) in the Python multiprocessing module: Queue (Queue / JoinableQueue) and Pipe. Among them, Queue supports the multi-producer, multi-consumer model, features process safety, and provides common methods such as put and get, as well as JoinableQueue 's task_done and join to simplify the notification mechanism for task completion;Pipes provide a lower-level communication method with higher efficiency than queues, making them suitable for bidirectional data exchange between two processes. This article demonstrates the usage of these two communication mechanisms through examples, and explains their respective application scenarios and precautions.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

12 Python Multiprocessing Programming - Process Pool

This document introduces the process pool (Pool) mechanism in the Python multiprocessing module. A process pool is used to manage a group of worker processes, avoiding the overhead caused by frequent process creation and destruction, and improving the efficiency of parallel task processing. The Pool provides synchronous methods (such as apply, map) and asynchronous methods (such as apply_async, map_async), which are respectively applicable to scenarios requiring blocking waiting for results or non-blocking parallel execution. Asynchronous calls support callback function, which can be combined with tools such as Progress Bar to realize real-time feedback on task completion.Through examples, this article demonstrates how to use synchronous and asynchronous calls, and explains the lifecycle management of process pools (including close, terminate, join) as well as their applicable scenarios.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

13 Python Asynchronous Programming - Basic Concepts, Event Loop and Callback Mechanism

This paper introduces the basic concepts and implementation approaches of Asynchronous Programming. Asynchronous Programming allows programs to continue executing other tasks while waiting for time-consuming operations (such as I/O or network requests) to complete, thereby improving efficiency. The article first distinguishes the concepts of sequence, concurrency, serial and parallel, pointing out that concurrency focuses on task scheduling while parallelism focuses on the actual simultaneous execution of tasks; asynchronous programming is a non-blocking execution model that can be applied to both single-threaded and multi-threaded environments.The core implementation mechanisms include the Event Loop and Callback. The event loop is responsible for managing tasks and the sequence of event triggering, while the callback function automatically processes results when a task is completed. The asyncio module in Python provides methods for managing event loops, coroutines, tasks, and Future objects, supporting an efficient asynchronous programming pattern.Through examples, this article demonstrates the execution process of asynchronous tasks and the calling method of the event loop, and showcases the application advantages of asynchronous programming in I/O-intensive or multi-task scenarios.

A Quick Guide to Python Parallel Computing: A Tutorial Summary

14 Python Asynchronous Programming - Management and Scheduling of Coroutines

This paper introduces the core concepts and implementation approaches of Python asynchronous programming and coroutines. A coroutine is a lightweight, non-preemptive task unit that can be suspended during execution and switch to other coroutines to achieve efficient task scheduling.This article elaborates on the roles of the Event Loop, Task, Future object and Callback function, and demonstrates how to use the asyncio module to manage coroutines and asynchronous tasks via async/await, create_task (), gather (), wait () and as_completed () to achieve concurrent execution of I/O-intensive programs and task timeout control.It demonstrates through examples the practical applications of coroutines in multi-task concurrency, task scheduling and result processing, as well as the differences and advantages of coroutines compared with traditional threads and processes.

A Quick Guide to Python Parallel Computing: A Tutorial Summary