Labs / MicroPython tutorials

20 File System and Storage Architecture - Draft Document Tutorial

Author: Li Zisheng · Source: FreakStudio open documentation, v1.2.0 · License: CC BY 4.0

Contents

1. Hardware Explanation

In previous lessons, we have written several MicroPython programs running on the Raspberry Pi Pico development board, and uploaded the written code to the development board (i. e., programmed the board) using the MicroPython plugin for PyCharm. After the code is uploaded, the MicroPython VM will compile it into bytecode, and the interpreter of the VM will interpret the bytecode line by line, convert it into low-level instructions, and interact with the hardware resources of the MCU.

So here comes the question: where exactly are these codes stored in the MCU, and how are they executed? We need to understand the relationship between the program and the MCU, and here we will first explain the knowledge points related to program storage.

1.1 Overview of the Memory Structure of Single-Chip Microcomputers

In a microcontroller, the storage space used to store program code and runtime data is referred to as memory. From the perspective of whether it is directly connected to the CPU, the memory of a microcontroller can be divided into the following two types:

1.1.1 Classification and Characteristics of Main Memory

Main memory can be divided into the following parts:

1.1.1.1 Introduction to ROM (Read-Only Memory)

ROM (Read-Only Memory) is a type of non-volatile memory that plays a crucial role in computers and embedded systems. ROM is characterized by data being written during memory manufacturing or initial programming, after which it can typically only be read and cannot be easily modified. The primary function of ROM is to store the firmware or programs required for system startup, ensuring that devices retain important data even after power is cut off.

The most common use of ROM is to store the firmware programs required when a device starts up, such as the BIOS of a computer or the boot program of an embedded device. Firmware is a set of instructions executed before the device runs, which is used to initialize hardware and load the operating system or other software; in embedded systems, ROM is often used to store the operating system or control code.

The original type of ROM is Mask ROM, whose data is directly written via a physical mask during the chip manufacturing process and cannot be modified after leaving the factory. Therefore, once the program changes, the entire chip has to be remanufactured. Later, there emerged another type of ROM that allows users to perform one-time programming with a dedicated device after the device leaves the factory — PROM, short for Programmable Read-Only Memory. Since it can only be programmed once, the chip has to be replaced once there is a writing error or a need to update the content. PROM is commonly used for testing during development and in low-volume production applications.

At present, the erasable and reprogrammable ROMs commonly used in embedded development fall into two categories:

The paging mechanism in EEPROM and the sector management in Flash both relate to the segmented management of storage media, yet there are still certain differences between them:

1.1.1.2 Introduction to RAM (Random Access Memory)

Random Access Memory (RAM) is a crucial storage medium in computers and other devices, which allows data to be read from and written to in any order. Its primary function is to provide fast-access data storage for the processor, supporting the operation of computer programs and the task scheduling of the operating system. However, it is a volatile memory, meaning that all data stored in RAM will be lost once the device is powered off.

20 File System and Storage Architecture - Draft Document Tutorial
RAM structure schematic diagram, in which the read/write control signal determines whether RAM performs a read operation or a write operation, the address decoder controls the word line (Wordline) and bit line (Bitline) to select specific storage units or rows based on the external input address signal, and the data input and output control circuit is responsible for inputting data into RAM or outputting data from RAM during the writing and reading process. The storage units are latches that can keep the stored data until it is refreshed or new data is written

According to their operating principles, RAM can be divided into two types:

20 File System and Storage Architecture - Draft Document Tutorial
Each inverter consists of a pair of transistors, and a total of 4 transistors are used for data storage. When writing data, the word line is pulled high to turn on the two access transistors, so that data is written into the inverter through the bit lines (Bit Line, BL and Bit Line Bar, BLB); during a read operation, the word line is pulled high to turn on the access transistors, the bit lines will be connected to the memory cell, and the voltage change on the bit lines will reflect the stored binary value according to the output state of the inverter, and the sense amplifier will detect the voltage change on the bit lines and decode the stored value.

Since no refresh operation is required, SRAM has a much faster read and write speed than DRAM, making it suitable for scenarios with high speed requirements; however, as each memory cell consists of 6 transistors, SRAM has a significantly lower storage density than DRAM, and the cost per bit of stored data is higher than that of DRAM.

1.1.2 Classification and Characteristics of External Memory

Common external memory includes external RAM, external ROM, external EEPROM and memory cards, as shown below:

20 File System and Storage Architecture - Draft Document Tutorial

Different types of external memory each have their own characteristics and application scenarios:

20 File System and Storage Architecture - Draft Document Tutorial

1.1.3 Address Space and Bus Architecture

Address space refers to the range of all memory that an MCU can directly access. Each memory location has a unique address, which is usually represented in hexadecimal format. For example, the RP2040 MCU onboard the Raspberry Pi Pico is a 32-bit microcontroller, so its address space is 32 bits wide, capable of representing 2^32 addresses, which amounts to 4,294,967,296 bytes (approximately 4GB) of space, ranging from 0x00000000 to 0xFFFFFFFF.

Here, the 4GB address space is divided into multiple functional regions, including ROM, SRAM, peripherals, XIP (Execute In Place), etc. The addresses in these regions are mapped to different storage and peripherals, making it convenient for developers to directly operate hardware resources, as shown in the following figure:

20 File System and Storage Architecture - Draft Document Tutorial

It mainly consists of the following parts:

Here, the APB and AHB-Lite buses serve as communication channels for connecting the CPU, memory, and peripherals (such as UART, SPI, I2C, etc.) to transmit data, addresses, and control signals. Through these buses, the CPU can access the addresses on the buses, control these peripherals, and read their statuses.

20 File System and Storage Architecture - Draft Document Tutorial

1.2 Storage Structure of Raspberry Pi Pico

Here, we first clarify several core concepts of "mapping"(which forms the foundation of memory access), before explaining the address mapping tables for memory and SRAM. For the resources inside a chip (including memory, registers, and peripherals) to be accessible, they must first be assigned unique "house numbers" — this is precisely the essence of "mapping":

For knowledge related to address mapping, register mapping, remapping and other related contents, please refer to the following link:

Fundamentals of STM32 Register Version — Memory Mapping:

https://developer.aliyun.com/article/935522

[Experience Sharing] STM32 Getting Started Series - STM32 Peripheral Address Mapping

https://shequ.stmicroelectronics.cn/thread-635602-1-1.html

Microcontroller Memory and Memory Mapping (Local Address, Logical Address, Global Address)

https://blog.csdn.net/u014157109/article/details/120370973

Memory access is entirely based on address mapping. The memory structure of the Raspberry Pi Pico consists of 16 kB of ROM with fixed content and 264 kB of SRAM that supports parallel access. The SRAM adopts a design of multiple physical memory blocks to improve access efficiency, and external Flash memory (2 MB in size) is accessed via the QSPI interface.

1.2.1 ROM

16 kB read-only memory (ROM) at address 0x00000000. The contents of the ROM are fixed once the chip is manufactured. It contains:

  1. Initialize the boot routine
  2. Flash Boot Sequence
  3. Flash Programming Routine
  4. USB mass storage device supporting UF2
  5. Utility libraries, such as fast floating-point arithmetic

The ROM bootloader can be downloaded at this link: https://github.com/raspberrypi/pico-bootrom

This ROM supports single-cycle read-only bus access and is connected to a dedicated AHB-Lite arbiter, enabling simultaneous access with other memory devices. Any attempt to write to the ROM will have no effect and will not trigger a bus fault.

1.2.2 SRAM

The on-chip SRAM totals 264 kB, which is physically divided into six banks, a design that significantly boosts the memory bandwidth for multiple bus masters — this allows multiple bus masters (such as Core0, Core1, and DMA) to access different banks simultaneously, eliminating resource contention — while software can treat it as a single 264 kB memory region for read and write operations (physical access to individual banks is implemented via address mapping).

There are no restrictions on the content stored in each bank: it can hold applications, data buffers, etc.

A bank can be divided into four 16k×32-bit blocks (each of 64 kB) and two 1k×32-bit blocks (each of 4 kB).

Each SRAM bank can be accessed via a dedicated AHB-Lite arbiter, which means different bus masters can access separate SRAM banks in parallel, enabling up to four 32-bit SRAM accesses (one per master) per system clock cycle.

In the memory mapping, the start address of the SRAM is 0x20000000, and the base address of each specific bank is as follows:

20 File System and Storage Architecture - Draft Document Tutorial

We can see that this mapping table contains the following information:

Here, the SRAM memory address mapping of the RP2040 adopts a striped storage mechanism to improve parallel access performance, and memory addresses are distributed across four memory blocks in units of words: as the system address increments by 4 (since 32 bits = 4 bytes), it will polling different banks in the order of "Bank0 → Bank1 → Bank2 → Bank3", then return to Bank0 to increment the "word address" — for example:

The advantage of this mapping is that when multiple modules access different system addresses simultaneously, they are actually accessing different physical banks. For example, when Core0 accesses 0x20000000 (Bank0) and DMA accesses 0x20000004 (Bank1), the two will not interfere with each other, realizing "parallel access" and greatly improving the read and write speed of SRAM.

Since consecutive bytes in the system address space are allocated to different memory blocks, multiple memory accesses can be performed simultaneously, which improves memory bandwidth and overall performance, reduces conflicts between different memory access requests, and thus enhances memory access efficiency.

Striped storage mechanism

The core concept of striping storage mechanism is to evenly distribute data across multiple storage blocks to achieve more efficient parallel access. Specifically, data is split into fixed-size blocks (usually bytes or words), which are allocated to different storage units (such as memory blocks, disk stripes, etc.) according to a certain rule.

Striped storage enables simultaneous access to multiple storage blocks, thereby improving overall bandwidth and system performance: for instance, in memory, striping allows multiple memory access requests to be processed concurrently, boosting data transfer speeds. Striped storage is also applied in disk arrays; in disk array systems such as RAID 0, striping is used to distribute data across multiple hard drives to enhance the read and write throughput of the disk.

In addition, the two 4kB regions located at 0x20040000 and 0x20041000 are directly mapped to two small 4kB memory blocks. These regions can be dedicated to per-core operations, such as stacks or frequently executed code, to ensure that no processor waiting occurs when accessing these regions.

In addition to the on-chip SRAM, there are two dedicated RAM blocks that can be used in certain scenarios:

Layman's Explanation of Dedicated RAM Blocks

Simply put, they are "part-time RAM": they have their own dedicated primary functions at ordinary times, and can "fill in" as regular memory when not in use, acting as a "backup storage area" within the chip.

  • First, let’s clarify the "core responsibilities" of the two dedicated RAMs:
    • Flash XIP cache: As mentioned earlier, Flash is an "external storage" device, and its speed is much slower than that of on-chip SRAM. The XIP (Execute In Place) function temporarily stores frequently used code and data from the Flash in a high-speed RAM (cache) — this way, when the kernel accesses these contents, it can directly read from the cache instead of waiting for the slow Flash, which is the core function of a cache.
    • USB data DPRAM: DPRAM is Dual-Port RAM, the dedicated data buffer of the USB module — when USB transfers data, it first stores the data in this RAM, which is then processed by the core or DMA; this is its core function.
  • There are two scenarios where memory can be "repurposed as regular RAM": These two RAM modules are normally occupied by their intended functions, but when the corresponding features are not in use, they can be utilized as standard memory (the address mapping also assigns them "spare address entries" for this purpose):
    • Disable Flash XIP cache to convert it to regular RAM: If you do not need the XIP acceleration feature for Flash (for example, when the entire program is stored in on-chip SRAM), you can disable the XIP cache — this block of RAM originally dedicated to Flash acceleration will then be freed up. Its "alternate base address" is 0x15000000, which you can use to store regular programs and data, effectively adding an extra block of memory.
    • When USB is not in use → USB DPRAM becomes general-purpose 4kB memory: If the project you are working on does not require USB functionality (e. g., a pure sensor acquisition device), the USB module will be left idle — its dedicated DPRAM (4kB) will also be freed up, with the corresponding "reserved memory address" being 0x50100000, and this 4kB RAM can then be used as general-purpose memory (e. g., for storing small data buffers or temporary variables).

Simply put, this is a "resource multiplexing" design for chips: the RAM dedicated to specific functions is opened up as general-purpose memory when the corresponding functions are idle, so that the limited storage resources can be utilized more flexibly.

1.2.3 External Flash

Here, we first introduce two key concepts:

20 File System and Storage Architecture - Draft Document Tutorial

The flash management of the RP2040 microcontroller adopts the Execute-In-Place (XIP) mechanism, which achieves efficient flash access through four main components: the QSPI interface, XIP hardware, XIP cache, and Serial Flash Interface (SSI).

20 File System and Storage Architecture - Draft Document Tutorial

The core of the entire structure is to enable external Flash to be accessed in the same way as internal memory, accelerate the access speed with cache, and ensure stable communication:

The external Flash (with a maximum support of 16MB) can be accessed via the QSPI interface using execute-in-place (XIP) hardware. This allows the system to address and access the external flash just as it would access internal memory, without the need to first copy code into internal memory: accesses to the 16MB flash region starting at the address range 0x10000000 will be mapped to read operations on the external serial flash, with the results directly returned to the master device that initiated the request.

The workflow of the XIP subsystem is as follows:

  1. System Access Decoding: When the system performs memory accesses, these accesses are routed through the main AHB-Lite slave, and the accesses are decoded to determine whether they are XIP accesses, direct accesses to the SSI (Serial Peripheral Interface)(e. g., for configuration), or accesses to other hardware and control registers within the XIP subsystem
  2. XIP Access Processing :
    • Cache lookup: For XIP access, the requested data is first looked up in the internal cache, which is used to accelerate access to recently accessed data. If the requested data is found in the cache (cache hit), the data is returned directly from the cache.
    • Cache miss: If the requested data is not found in the cache (i. e., a cache miss occurs), an external serial access request will be generated, which is sent to the external flash memory via the SSI interface to obtain the required data
  3. Data storage and forwarding:
    • Data Storage: Data retrieved from external flash memory is stored in the cache to enable faster data delivery for future accesses.
    • Data forwarding: The acquired data is then forwarded to the system bus to fulfill the original access request. In this way, the XIP subsystem ensures that the accessed data is available on both the system bus and the processor.

1.2.3.1 XIP Cache Mechanism

XIP cache is an internal cache designed to accelerate access to external flash memory. It has a size of 16KB, adopts a two-way set-associative structure, and features a latency of 1 clock cycle per cache hit; this cache is exclusively used for accesses to XIP flash, and software does not need to account for cache consistency in most cases, with the exception of flash programming operations.

The cache is mapped to the 24-bit flash address space. In the system addresses of the RP2040, the upper 8 bits are used for segment decoding, and the remaining 24 bits are used for the flash address, so the maximum supported flash size is 16MB. The flash address space has multiple mirrors in the RP2040's address space, and each mirror has different cache behavior. The specific mirror addresses and their cache behaviors are as follows:

By setting the CTRL.EN register bit, the cache can be disabled. When the cache is disabled, all XIP address mirrors from 0x10 to 0x13 will bypass the cache and directly access the flash memory, which will significantly affect the execution performance of XIP code.

20 File System and Storage Architecture - Draft Document Tutorial
1.2.3.2 XIP Cache Flush and Maintenance

When the software reprograms the flash memory content, it is necessary to clear the stale data and code in the cache to ensure that the system uses the latest data without restarting. In this case, the FLUSH register can be used to refresh the entire cache content.

20 File System and Storage Architecture - Draft Document Tutorial

We can implement cache flushing by writing a 1 to the 0th data bit of the FLUSH register, and the cache flushing is accomplished by clearing the cache tag memory. Since the total cache size is 16KB, each line of the tag memory is 8 bytes, and there are 2 ways per set, the flushing process requires approximately 1024 clock cycles.

We can wait for the refresh operation to complete by reading the 0th data bit of the FLUSH register, which will block the processor until the refresh finishes; alternatively, we can check the refresh status by polling the STAT register.

20 File System and Storage Architecture - Draft Document Tutorial

Here, accessing flash memory data during a cache flush is safe: if another master device accesses the flash memory data while a cache flush is in progress, those master devices will be blocked until the cache flush is completed, which ensures that no inconsistent data access occurs before the flush finishes.

It should be noted that when the XIP cache is used as a 16KB SRAM (with the mirror address being 0x15…), the cache will be directly used as memory and support both read and write operations. If the cache is flushed at this time, the data content in the cache will not be modified; however, if a write operation is performed on the address range where the cache acts as SRAM during the flushing process, data inconsistency or corruption may occur. To avoid data corruption, it is recommended to perform write operations on the cache used as SRAM only after the cache flushing operation is completed, which can be achieved by confirming that the cache flushing operation has been fully finished before executing the write operation.

Completing a full cache flush will significantly slow down the execution speed of subsequent code until the cache is "re-warmed" to restore normal performance. To avoid this performance penalty, an alternative approach is to only invalidate the cache contents for a specific address range.For example, when a sector of the flash memory is erased and reprogrammed, it is only necessary to ensure that the stale data of that sector in the cache is updated, rather than flushing the entire cache. In the RP2040, you can trigger the invalidation of a specified address range in the cache by writing data to the 0x10.. . mirror address. The 0x10.. . mirror address here refers to a special mapping of the cache, which supports partial invalidation of data within a specific address range:

  1. Lookup cache entries: When writing to the 0x10.. . mirror address, the cache checks for cache entries within this address range
  2. Delete entries: If matching cache entries are found, these entries will be deleted, the cached data will be marked as invalid, and when these addresses are accessed next time, the data will be reloaded from the data source (such as flash memory)

1.2.3.3 SSI Serial Flash Interface

SSI (Synchronous Serial Interface) is a critical peripheral in the RP2040 microcontroller, designed to handle serialized flash memory access. It supports various SPI (Serial Peripheral Interface) modes, including standard SPI, Dual SPI (DSPI), and Quad SPI (QSPI). SSI enables XIP (Execute-in-Place) access via multiple operating modes, ensuring compatibility with nearly all types of serial flash memory devices.

The maximum SPI clock frequency of the SSI is half of the system clock frequency. It supports inserting instruction prefixes or mode continuation bits during each XIP access, and also supports the standard FIFO (First-In, First-Out) SPI host mode, with DMA (Direct Memory Access) support provided at the same time, which makes it applicable to high-speed data transmission.

1.2.3.4 Flash Streaming and Auxiliary Bus Slave

In embedded systems, flash memory is typically much larger than static random access memory (SRAM). Therefore, when it is necessary to transfer a large amount of data from flash memory to RAM, adopting a streaming transmission approach is extremely useful. This operation allows DMA (Direct Memory Access) to stream data in the background, while the processor in the foreground continues to execute other tasks.

Standard XIP (Execute-in-Place) operations are not highly compatible with streaming transmission, because when the SSI (Serial Peripheral Interface) performs serial transmission, it will cause a relatively long bus delay for DMA. This delay may be acceptable for the processor, since the processor generally has no other valid operations to perform while waiting for instruction fetch, and code execution usually features a high cache hit rate.

However, for DMA, such delays will prevent other active DMA channels from continuing to operate, thereby reducing the overall DMA throughput, which calls for a solution to mitigate this impact.

In the RP2040, STREAM_ADDR and STREAM_CTR registers are used to configure the continuous data reading from the flash and the subsequent data transfer to the memory, an operation referred to as streaming, whose primary purpose is to efficiently load large volumes of data from the external Flash into the SRAM memory:

20 File System and Storage Architecture - Draft Document Tutorial
20 File System and Storage Architecture - Draft Document Tutorial

The specific steps are as follows:

Once streaming is configured, the XIP (Execute-in-Place) subsystem automatically handles these data read operations in the background without requiring the CPU to wait. During the streaming process, the streaming hardware of the XIP subsystem performs data reading at a lower priority. This means that streaming operations will not preempt the processor's executing tasks or cause significant latency.

When code is executed from flash, cache misses may occur (meaning there is no data in the cache and it needs to be read from flash). Compared with regular XIP access, streaming hardware will experience a short Cool Down (seven clock cycles) after a cache miss to avoid causing additional latency when a cache miss occurs.

Streaming reads data from the flash memory and pushes it into a small FIFO buffer queue. When the data in the FIFO is ready, it generates a DREQ (Data Request) signal, which notifies the DMA controller that data can be read from the FIFO. The DMA controller is responsible for fetching the data from the FIFO and transferring it to the target memory or device. It should be noted that the DMA only reads the FIFO after the data is ready, so it will not be blocked by the flash reading process.

20 File System and Storage Architecture - Draft Document Tutorial

Although the data is already prepared and stored in the FIFO, if another system master (such as a CPU or an external device) is accessing the XIP flash and gets blocked due to a cache miss or other reasons, this may affect the operation of the DMA. To address this issue, the RP2040 utilizes an auxiliary bus slave, which is a simple bus interface exclusively designed for accessing the streaming FIFO; this slave device is connected to the FASTPERI arbiter, which serves native AHB-Lite peripherals that do not generate wait states.This way, the DMA will not encounter additional wait states or blocking when accessing the FIFO, since the accesses of the auxiliary bus slave will not be interfered with by other bus operations.

1.2.3.5 Performance Counters

The XIP (Execute-In-Place) subsystem of the RP2040 is equipped with two performance counters, which are used to monitor and analyze the performance of the cache. These counters can help developers understand the usage and efficiency of the cache, so as to optimize code and system design.

Both counters are 32 bits wide, which means the maximum value they can count up to is 0xFFFFFFFF (i. e., 4294967295); once this value is reached, the counter will wrap around (i. e., return to 0). Meanwhile, the value of the counter can be cleared by writing any arbitrary value, which is typically used to reset the counter to start a new counting cycle.

20 File System and Storage Architecture - Draft Document Tutorial
20 File System and Storage Architecture - Draft Document Tutorial

When used together, these two counters can provide useful performance data related to XIP operations. For example, by comparing the number of hits with the total number of accesses, the cache hit rate can be calculated, so as to understand the effectiveness of the cache and the room for optimization.

1.3 Memory Areas of MicroPython

When learning MicroPython, understanding its "memory regions" is equivalent to figuring out where our programs "reside" and in which "cabinet" different types of data are stored. Only by grasping the rules governing data storage can we avoid writing code that causes memory overflow or unexplained data errors, and also gain insight into the underlying logic of program execution.

First, let’s supplement two basic concepts that every beginner must know:

The storage area distribution of MicroPython is similar to the memory model of traditional programming languages (C/C++), which is mainly divided into two major blocks: Flash and SRAM (Static Random Access Memory). Each block contains sub-areas with different functions, each performing its own duties:

1.4 File System in MicroPython

1.4.1 Concept and Composition Structure of File System

A file system is the structure and method used by an operating system to manage files on storage media such as hard disk drives, solid-state drives, and flash memory. It is responsible for file creation, reading, writing, deletion, as well as directory organization and management, enabling users and applications to access and manage data in a structured manner.

You can think of a "file system" as the "organization rules" that a device (such as a USB flash drive or the storage on an embedded development board) uses to store files — just like how you set rules for your drawers, like "which layer holds documents and which holds photos", a device relies on the file system to manage "where to store a file, how to locate it, and how to store it more efficiently".

20 File System and Storage Architecture - Draft Document Tutorial

Typically, a file system has the following functions:

FAT is one of the simple and easy-to-understand file systems (commonly used in embedded devices). First, let’s supplement a basic concept that every beginner must know:

Taking the FAT file system as an example, a complete file system usually consists of the following parts:

20 File System and Storage Architecture - Draft Document Tutorial
The file system divides disk space into several clusters to facilitate the management and organization of stored data. A cluster is a basic storage unit, each cluster contains one or more sectors, and each sector is the smallest storage unit in a storage device. The size of a cluster may vary across different file systems and storage devices; for FAT12/FAT16, the cluster size is typically 512 bytes, 1KB, 2KB, or 4KB.

Common file system types include FAT (FAT12, FAT16 and FAT32), NTFS, ext, LittleFS, etc. Among them, the FAT file system is a commonly used file system for embedded devices, featuring simple design, easy implementation and understanding, and low overhead. However:

In MicroPython, the three most commonly used file systems are FAT, LittleFS v1 and LittleFS v2, and their comparison is as follows:

20 File System and Storage Architecture - Draft Document Tutorial

1.4.2 VFS Virtual File System

First, let’s cover 2 basic concepts that every beginner must know to help you quickly understand VFS:

In MicroPython, VFS (Virtual File System) is a module designed to provide file system support for embedded devices. As an abstraction layer, it offers a unified file system interface:

Additionally, users can mount different types of file systems to specific paths and unmount them when necessary: you can think of "mounting" as "adding an access point" for a storage device — for example, "connecting" an SD card to the /sd folder in the system, after which you can operate the SD card simply by accessing /sd; "unmounting", on the other hand, means safely "removing" this access point to prevent data corruption.

For example, we can mount the FAT file system of an SD card to /sd, and at the same time mount a SPIFFS file system to /spiffs, and manage files through a unified file operation interface; through the VFS module, we can also create a block device (a device that performs read and write operations in units of blocks) at the software level to simulate the behavior of a storage device. For instance, we can create a virtual RAM block device in an embedded system or operating system.

VFS provides the following methods for mounting and unmounting file systems:

VFS only acts as a "manager", you need to first create a file system object of the corresponding type (such as FAT / LittleFS) before you can mount and use it. Below are the common file system classes (corresponding to the class descriptions you provided).

The following are file system implementation classes, which can be used to build file systems such as FAT, LittleFS v1, and LittleFS v2:

20 File System and Storage Architecture - Draft Document Tutorial
20 File System and Storage Architecture - Draft Document Tutorial

As shown below:

The following table shows the file systems included in the MicroPython firmware for different chips by default:

20 File System and Storage Architecture - Draft Document Tutorial

1.4.3 Block Devices

A file system is built on top of block devices, which are a type of storage device that divides the storage space into fixed-size blocks and allows random access to these blocks — meaning each block has a unique address and can be read from or written to independently. Put simply, block devices store data in blocks, and the file system organizes these blocks into files and directories, enabling users to access and manage data conveniently.

20 File System and Storage Architecture - Draft Document Tutorial

Character Devices and Stream Devices

Character devices perform data transmission in units of characters, do not support random access, and usually conduct data reading and writing in a sequential manner; for example, serial communication peripherals, keyboards, and mice are all character devices, which are often used for streaming data transmission, real-time data input and output and other such requirements.

A "Stream Device" is an extended or supplementary concept of character devices, mainly used to process continuous, streaming data streams. The data transmission methods of these devices are usually similar to those of character devices, but they focus on meeting the needs of dynamic data stream processing. Common stream devices include network streams (such as network interface cards), audio streams (such as microphones and speakers), video streams (such as cameras), Streaming Media devices (such as webcams, Real Time Communication transmission devices), etc.

MicroPython does not provide operation methods for block devices, but it offers an abstract block device interface vfs.AbstractBlockDev class. This template specifies the methods that a block device must implement (such as "read data from a certain block" and "write data to a certain block"), which is equivalent to the "format of an answer sheet for an exam". Only by writing code in accordance with this template can MicroPython recognize it as a block device and then create a file system on it.

The actual block device class must implement the methods described below:

20 File System and Storage Architecture - Draft Document Tutorial

2. Software Control Method

2.1 os Module

The os module provides the following methods for operations such as file reading and writing, directory creation, etc. :

Method Name

input parameters

Return Parameters

Function Description

os.uname()

None

(sysname, nodename, release, version, machine) tuple

Returns a tuple containing the system name, node name, system version, MicroPython version, and hardware identifier.

os.urandom (n)

n (number of random bytes to generate)

bytes object

Returns a bytes object containing n random bytes, generated using a hardware random number generator whenever possible.

os.chdir(path)

path (path, string type)

None

Change the current working directory to the specified path.

os.getcwd()

None

Current working directory path (string)

Returns the path of the current working directory.

os.ilistdir ([dir])

dir (optional, path, string type)

An iterator that returns (name, type, inode[, size]) tuples

Lists the contents of the specified directory (or the current directory) and returns an iterator of tuples containing the name, type, inode, and (optionally) size.

os.listdir ([dir])

dir (optional, path, string type)

Table of Contents Content List (List Type, String)

Lists the contents of the specified directory (or the current directory) and returns a list of the names of files and directories.

os.mkdir(path)

path (path, string type)

None

Create a new directory with the path path.

os.remove(path)

path (file path, string type)

None

Delete the file at the specified path.

os.rmdir (path)

path (directory path, string type)

None

Delete the empty directory at the specified path.

os.rename

(old_path, new_path)

old_path (old path, string type)
new_path (new path, string type)

None

Rename the file or directory from old_path to new_path.

os.stat(path)

path (file/directory path, string type)

File or directory status information (tuple)

Obtain the status information of a file or directory, such as file size, permissions, modification time, etc.

os.statvfs(path)

path (file system path, string type)

File System Status Information (Tuple)

Returns status information of the file system, such as block size, available blocks, number of files, etc.

os.sync()

None

None

Synchronizes all changes on the file system to the disk. Typically, file operations such as write () or close () write data to the cache, and the data is only written to the actual storage medium under specific conditions (for example, when the file is closed or the cache is full).

os.dupterm

(stream_object, index=0)

stream_object (stream object)
index (integer, optional, default value is 0)

the previous stream object or None

Retarget the terminal's input and output to the specified stream_object stream object. Multiple slot index are supported; pass None to cancel the retargeting.

2.2 vfs Virtual File Module

vfs (virtual file system) module contains the functions for creating file system objects as well as mounting and unmounting these file systems.

The functions of the file system mounting are as follows:

Method Name

input parameters

Return Parameters

Function Description

vfs.mount(fsobj, mount_point, *, readonly)

- fsobj : File system object or block device.
- mount_point: Mount Point (root directory '/' or subdirectory '/<name>' ).
- readonly (optional): Whether to mount in read-only mode.

None

Mount the file system object fsobj to the specified location mount_point in the VFS; if it is a block device, the file system type will be automatically detected, and mounting to the root directory or a subdirectory is supported.

vfs.umount(mount_point)

- mount_point: the mount location to be unmounted, or the mounted file system object.

None

Unmount the file system mounted at the specified location, which supports specification via a Mount Point or file system object; during the unmount process, the umount () method of the file system object will be called.

The following are the mountable file system types:

Class Name

Input parameters

Return Parameters

Function Description

vfs.VfsFat(block_dev)

- block_dev: Block device object, which provides storage for the FAT file system.

file system

object

Create a file system object formatted with the FAT file system, where block_dev provides storage, and the created object can be mounted via the mount () method.

vfs.VfsLfs1

(block_dev, readsize=32, progsize=32, lookahead=32)

- block_dev : Block device object, which provides storage for the littlefs v1 file system.
- readsize (optional): Read size, default value is 32.
- progsize (optional): Programming size, default value is 32.
- lookahead (optional): Look-ahead read size, default value is 32.

file system

object

Create a file system object using the littlefs v1 file system format, block_dev provides storage, and the created object can be mounted via the mount () method.

vfs.VfsLfs2(block_dev, readsize=32, progsize=32, lookahead=32, mtime=True)

- block_dev : Block device object, providing storage for the littlefs v2 file system.
- readsize (optional): Read size, default value is 32.
- progsize (optional): Program size, default value is 32.
- lookahead (optional): Lookahead buffer size, default value is 32.
- mtime (optional): Whether to enable file modification timestamps.

file system

object

Create a file system object using the littlefs v2 file system format, with block_dev providing storage; this implementation supports file modification timestamps, and allows timestamps to be enabled or disabled dynamically.

vfs.VfsPosix(root=None)

- root (optional): A path in the host file system that serves as the root directory for the VfsPosix object; if left unspecified, the current directory will be used.

file system

object

Creates a filesystem object to access the host's POSIX filesystem. If root is specified, it uses that path as the root directory; otherwise, it uses the host's current directory.

2.3 Block Device Abstract Class

vfs.AbstractBlockDev is an abstract class for implementing block devices, and concrete block device classes must implement the methods described below:

Method Name

input parameters

Return Parameters

Function Description

readblocks(block_num, buf)

- block_num : the block number being read
- buf: the byte array that stores the data

None

Starting from the specified block_num, read multiple blocks and store the data into buf;The length of buf, which is a multiple of the block size, indicates the number of blocks to be read.

readblocks

(block_num, buf, offset)

- block_num : the block number being read
- buf : the byte array that stores the data
- offset: the byte offset within the block

None

Read any number of bytes starting from the specified block_num and offset positions, and store the data into buf;The length of buf indicates the number of bytes to be read.

writeblocks(block_num, buf)

- block_num : the block number to be written
- buf: the byte array of the data to be written

None

Writes the data in the buf to the block starting from the block_num . The number of blocks written is determined by the length of the buf , which is a multiple of the block size. Buf .

writeblocks

(block_num, buf, offset)

- block_num : the block number to be written
- buf : the byte array of the data to be written
- offset: the byte offset within the block

None

Write the data in buf to the blocks starting from block_num, starting the write operation at the offset position; the number of bytes to be written is determined by the length of buf.

ioctl (op, arg)

- op : Opcode (e. g., initialization, synchronization, etc.)
- arg: Argument, with different meanings depending on the opcode

Depending on the opcode, this may be an integer, None, or an error code

Controls a block device and executes the functions specified by different operation codes. Common operations include: initializing the device (op= 1), acquiring the number of blocks (op= 4), acquiring the block size (op= 5), erasing blocks (op= 6), synchronizing the device (op= 3), etc.

3. Application Examples

3.1 File Read and Write Operations

In the following code, we use the os module in MicroPython to implement a file read and write routine. This code reads data from the ADC internal temperature sensor, writes the data to a file, and simultaneously reads the data from that file.

The following code can be found in the elegance-devkit v1\Demo\70 FileSys_WriteRead folder in the resource package we provide.

The sample code is shown below:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/16 11:58 AM
# @Author  : Li Qingshui
# @File    : main.py       
# @Description : File system experiment, using the os module to read and write files and redirect the terminal

# ======================================== Import Modules =========================================

# Import the operating system module, which provides file operations
import os
# Import the hardware module
from machine import ADC
# Import time-related modules
import time

# ======================================== Global Variables ============================================

# Define the file path used to store the temperature sensor data
file_path = "temperature_data.txt"

# ======================================== Functions ============================================

def read_temperature() -> float:
    """
    Reads the ADC value and converts it to a temperature in degrees Celsius; applies to the Raspberry Pi Pico internal temperature sensor.

    Implementation steps:
    1. Read the raw 16-bit ADC value
    2. Convert it to a 0-3.3V voltage value
    3. Apply the conversion formula for the RP2040 chip's built-in temperature sensor
    4. Round the value

    Args:
        None

    Returns:
        float: The converted temperature value in degrees Celsius, with two decimal places

    Raises:
        OSError: Raised when the ADC read fails
        ValueError: Raised when an invalid temperature value is calculated
    """

    # Declare global variables
    global sensor_temp

    # Read the ADC value; the ADC range is 0~65535, and the supply voltage is assumed to be 3.3V
    adc_value = sensor_temp.read_u16()
    # Convert the ADC value to a voltage; the ADC has 16-bit resolution
    voltage = adc_value * 3.3 / 65535
    # Use the Raspberry Pi Pico temperature formula to convert the voltage to a temperature (this is the conversion formula for the Pico's internal temperature sensor)
    temperature = 27 - (voltage - 0.706) / 0.001721

    # Keep two decimal places
    temperature = round(temperature, 2)
    # Return the temperature value
    return temperature

# ======================================== Custom Classes ============================================

# ======================================== Initialization ==========================================

# Power-on delay
time.sleep(3)
# Print debug information
print("FreakStudio : Save temperature data to file system")

# Initialize the internal temperature sensor
sensor_temp = ADC(4)

# ========================================  Main Program  ===========================================

# Open the file in write mode; create it if it does not exist
file = open(file_path, "w")
# Get the initial time when the program started running
start_time = time.ticks_ms()

# Print debug information
print("Start to record temperature data")
# Start recording temperature data and write it to the file
for i in range(10):
    # Read the current temperature
    temp = read_temperature()
    # Record the current run time
    current_time = time.ticks_ms()
    # Calculate the run time interval
    time_diff = time.ticks_diff(current_time, start_time)
    # Print debug information indicating that writing is in progress
    print("File Writing...")
    # Write the recorded time difference (in milliseconds) and the temperature to the file
    file.write("Time: {} ms, Temperature: {:.2f} °C\n".format(time_diff, temp))
    # Sync the file system after each write to make sure the data is saved
    os.sync()
    # Sleep for 10ms
    time.sleep_ms(10)

# Close the file
file.close()
# Print debug information
print("File Writing Finished")

# To verify the file contents, reopen the file, read it and display it
# Reopen the file in read-only mode
file = open(file_path, "r")
# Print debug information
print("File Read Start :")

# Read the file contents line by line
for line in file:
    # Print the file contents to the console
    print(line.strip())

# Close the file
file.close()

Here, we first use ADC channel 4 to access the internal temperature sensor, and define a read_temperature function that can convert ADC values to voltage, and then convert the voltage to Celsius temperature using the temperature conversion formula provided by Raspberry Pi Pico; after that, we create a file named temperature_data.txt and operate it in write mode.

In the main program section, perform the following operations:

  1. Open the file temperature_data.txt in write mode; record the initial time when the program starts running via time.ticks_ms (), which is used to calculate the time difference subsequently.
  2. Loop 10 times, read the current temperature data and record the time difference each time, write the time difference and temperature data into a file in text format, and use os.sync () to synchronize the file system to ensure data security.
  3. After completing data writing, close the file file.close () to ensure proper release of resources.
  4. Reopen the file in read-only mode temperature_data.txt, read the file content line by line, and use print () to display the content on the Console for verification. Finally, close the file to complete the program execution.

Here, it should be noted that os.sync () function is used in MicroPython to synchronize the cached data of the file system, that is, to force the data in the buffer that has not yet been written to the disk to be written to the file system; normally, file operations (such as write () or close ()) will write data to the cache, and only under specific conditions (such as when a file is closed or the cache is full) will the data be written to the actual storage medium.

In embedded systems, especially devices using file systems such as flash memory and SD cards, unsynchronized file data may be lost or corrupted if the system is suddenly powered off. If a program is performing a long-term file write operation, for data security considerations, it can periodically call os.sync () to ensure that the data written previously will not be lost even if the program is interrupted midway.

Burn the code, open the terminal, and the following content will be displayed:

20 File System and Storage Architecture - Draft Document Tutorial

3.1 Application Experiment.mp4 3.2 MB

As you can see, after we write the time difference and temperature data to the file temperature_data.txt, we can successfully read the content in the file and print it to the terminal.

After the code finishes executing, we use the function os.listdir () to list the current directory, and we can see that the file temperature_data.txt already exists in the file system; we can also use the os.statvfs () function to read the file status, which returns a tuple containing file system information:

Here, multiplying f_bsize and f_blocks yields the size of the internal file system of the Raspberry Pi Pico as 1441792 bytes, which is 1.37 MB.

3.2 Custom Logging Function

In the following code, we use a custom class to implement a logging module that supports the following features:

20 File System and Storage Architecture - Draft Document Tutorial

The sample code is as follows:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/16 2:40 PM
# @Author  : Li Qingshui
# @File    : logger.py       
# @Description : Logging module with level classification, log filter and formatter features

# ======================================== Import Modules =========================================

# Import the time module, used to get the current time
import time
# Import the os module, used for file operations
import os
# Import the threading module and locks
import _thread
# Import system-related modules
import sys
# Import the micropython module, used to optimize code
import micropython

# ======================================== Global Variables ============================================

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

# Define the log levels
class LogLevel:
    DEBUG    = 1 # Debug level
    INFO     = 2 # Info level
    WARNING  = 3 # Warning level
    ERROR    = 4 # Error level
    CRITICAL = 5 # Critical error level

# Logger class
class uLogLite:
    """
    Logger class that provides multi-level logging, filtering and formatting, and supports file rotation and thread-safe operation.

    This class encapsulates the complete logging process, including log level filtering, message formatting, file output and rotation management.
    It supports flexible log filtering through a filter chain, and the log output style can be customized through a formatter.

    Attributes:
        name (str): Logger identifier name, used to generate the log file name prefix
        level (int): Current log level (LogLevel enum value)
        filters (List[Callable[[int, str], bool]]): List of log filter functions
        formatter (Optional[object]): Log formatter instance; must implement the format(level, message) method
        output (str): Output target. "terminal" means terminal output; any other string is a folder path. Only single-level folder paths are supported here.
        max_logs (int): Maximum number of log entries per log file
        rotate_interval (int): Log rotation time interval (unit: hours)
        log_count (int): Number of log entries recorded in the current file
        last_rotation_time (float): Timestamp of the last rotation (UNIX timestamp)
        lock (Lock): Thread mutex lock object
        log_file (Optional[TextIO]): Current log file handle

    Methods:
        __init__(self, name: str, level: int = LogLevel.DEBUG, output: str = "terminal",
                max_logs: int = 1000, rotate_interval: int = 24) -> None:
            Initialize the logger instance

        add_filter(self, log_filter: Callable[[int, str], bool]) -> None:
            Add a log filter function

        set_formatter(self, formatter: object) -> None:
            Set the log formatter instance

        log(self, level: int, message: str) -> None:
            Base log recording method

        debug(self, message: str) -> None:
            Record a DEBUG-level log

        info(self, message: str) -> None:
            Record an INFO-level log

        warning(self, message: str) -> None:
            Record a WARNING-level log

        error(self, message: str) -> None:
            Record an ERROR-level log

        critical(self, message: str) -> None:
            Record a CRITICAL-level log

        exception(self, exc: Exception, terminal_display: bool = True) -> None:
            Record exception stack trace information

        close(self) -> None:
            Safely close the log file

    Private Methods:
        _open_log_file(self) -> None:
            Open/create the log file and initialize the counter

        __is_single_level_path(self, path: str) -> bool:
            Check whether the path is a valid single-level folder path (for the MicroPython file system)

        _apply_filters(self, level: int, message: str) -> bool:
            Apply all registered filters

        _format_message(self, level: int, message: str) -> str:
            Format the log message

        _write_log(self, formatted_message: str) -> None:
            Thread-safe log write operation

        _should_rotate(self) -> bool:
            Check whether the rotation condition is met

        _rotate_log_file(self) -> None:
            Perform log file rotation
    """
    def __init__(self, name: str, level: int = LogLevel.DEBUG, output: str = "terminal",
                 max_logs: int = 1000, rotate_interval: int = 24) -> None:
        """
        Logger initialization

        Args:
            name (str): Logger identifier name
            level (int): Default log level (LogLevel enum)
            output (str): Output target, "terminal" or a folder path. Only single-level folder paths are supported here.
            max_logs (int): Maximum number of log entries
            rotate_interval (int): Log rotation time interval (hours)

        Raises:
            OSError: Raised when file creation fails, name is empty or an empty string, level is not in the LogLevel enum, or output is not a string.
        """
        # Check whether name is empty or an empty string
        if not name or name == "":
            raise ValueError("name cannot be empty")

        # Check whether name is a string
        if not isinstance(name, str):
            raise TypeError("name must be a string")

        # Check whether level is an integer and a value in LogLevel
        if not isinstance(level, int) or level not in [getattr(LogLevel, attr) for attr in dir(LogLevel) if
                                                       not attr.startswith('__')]:
            raise TypeError("level must be an integer of LogLevel enum")

        # Check whether output is valid
        if not isinstance(output, str):
            raise TypeError("output must be a string")

        # Verify that output is only a single-level folder path
        if output != "terminal":
            if not self._is_single_level_path(output):
                raise ValueError(
                    "Output path must be a single-level directory\n"
                    "Examples: 'logs', '/data', 'mylog'"
                )

        # Initialize the logger; accepts the log name, log level, and output method (file or terminal)
        self.name = name
        self.level = level
        # Initialize the filter list
        self.filters = []
        # Initialize the formatter
        self.formatter = None
        # Log output method: can be "terminal" or a folder path
        self.output = output
        # Maximum number of log entries
        self.max_logs = max_logs
        # Rotation time interval (hours)
        self.rotate_interval = rotate_interval

        # Initialize the log entry counter
        self.log_count = 0
        # Record the last rotation time
        self.last_rotation_time = time.time()

        # Create the mutex lock
        self.lock = _thread.allocate_lock()

        # Open the log file
        self._open_log_file()

    @staticmethod
    def _is_single_level_path(path: str) -> bool:
        """
        Check whether the path is a valid single-level folder path (for the MicroPython file system)

        Single-level folder paths support the following formats:
            "logs"    - Relative path under the current directory
            "/logs"   - Absolute path under the root directory
            "" or "." - Current directory
            "/"       - The root directory itself

        Args:
            path (str): Path string to check

        Returns:
            bool: Returns the check result:
                - True:  Valid single-level path
                - False: Multi-level path or contains illegal characters
        """
        # Empty path or current directory
        if not path or path == "." or path == "/":
            return True

        # Normalize the path (handle leading/trailing slashes)
        normalized = path.strip("/")

        # Check whether it contains a subpath separator
        return "/" not in normalized and "\\" not in normalized

    @micropython.native
    def _open_log_file(self) -> None:
        """
        Create/open the log file. If the file already exists, read its line count to initialize the counter; for a new file the counter starts from 0.

        Args:
            None

        Returns:
            None

        Raises:
            OSError: Raised when file creation fails

        Notes:
            - File name format: {name}_{year-month-day_hour-minute-second}.txt
            - Automatically falls back to terminal output on failure
        """
        # Get the current time: format is (year, month, mday, hour, minute, second, weekday, yearday)
        t = time.localtime()
        # Record the timestamp and format it as a year-month-day_hour-minute-second string (e.g., 20240917_143001)
        timestamp = "{:04}{:02}{:02}_{:02}{:02}{:02}".format(t[0], t[1], t[2], t[3], t[4], t[5])

        # If output is to the terminal
        if self.output == "terminal":
            # Set the log file handle to None; terminal mode needs no file operations
            self.log_file = None
            self.log_count = 0
        else:
            # Generate the log file name from the name and timestamp: name_timestamp.txt
            log_filename = f"{self.output}/{self.name}_{timestamp}.txt"

            # Try to create the directory (only single-level directories are handled)
            try:
                os.mkdir(self.output)
            except OSError:
                # Ignore if the directory already exists
                pass

            try:
                # Check whether the file already exists
                try:
                    os.stat(log_filename)
                    file_exists = True
                except OSError:
                    file_exists = False

                # Open the file in append mode (created if it does not exist)
                self.log_file = open(log_filename, 'a')

                # Line count
                if file_exists:
                    count = 0
                    with open(log_filename, 'r') as f:
                        while True:
                            # Read in 128-byte chunks (suited to memory-constrained environments)
                            chunk = f.read(128)
                            # Reached the end of the file
                            if not chunk:
                                break
                            # Count the newline characters (one per line)
                            count += chunk.count('\n')
                    self.log_count = count
                else:
                    # If it is a newly created file, initialize the counter
                    self.log_count = 0
            except OSError as e:
                # Failed to open the file; print the error message
                print(f"无法打开文件 {log_filename}: {e}")
                # Set the file handle to None
                self.log_file = None

    def add_filter(self, log_filter: callable[[int, str], bool]) -> None:
        """
        Add a log filter

        Args:
            log_filter (callable): Filter function; receives (level, message) and returns bool

        Returns:
            None
        """
        # Add the log filter
        self.filters.append(log_filter)

    def set_formatter(self, formatter: object) -> None:
        """
        Set the log formatter

        Args:
            formatter (object): Object that implements the format(level, message) method
            Must have a format(level, message) method for formatting log messages

        Returns:
            None

        Raises:
            TypeError: Raised when the formatter does not implement the format(level, message) method
        """
        # Check whether the formatter implements the format(level, message) method
        # hasattr(formatter, "format") checks whether formatter has a format attribute
        # callable(getattr(formatter, "format")) ensures format is a callable method rather than a plain attribute
        if not hasattr(formatter, "format") or not callable(getattr(formatter, "format")):
            raise TypeError("formatter must implement a callable format(level, message) method")

        # Set the log formatter
        self.formatter = formatter

    def _apply_filters(self, level: int, message: str) -> bool:
        """
        Apply all registered filters. If any filter returns False, the log is not recorded.

        Args:
            level   [int]: Log level
            message [str]: Log message

        Returns:
            bool: Passes when all filters return True
        """
        # Apply the log filters; if any filter returns False, the log is not recorded
        for log_filter in self.filters:
            if not log_filter(level, message):
                return False
        return True

    @micropython.native
    def _format_message(self, level: int, message: str) -> str:
        """
        Format the log message. If a formatter is set, use it; otherwise use the default format.

        Args:
            level   [int]: Log level.
            message [str]: Log message.

        Returns:
            str: The complete formatted log entry.
        """
        # If a formatter is set, use it; otherwise use the default format
        if self.formatter:
            return self.formatter.format(level, message)
        return f"{self.name} - {level}: {message}"

    @micropython.native
    def _write_log(self, formatted_message: str) -> None:
        """
        Thread-safe log write operation. Choose the write method according to the log output method; it can write to a file or the terminal.

        Execution flow:
            1. Acquire the thread lock
            2. Check the rotation condition
            3. Perform the actual write
            4. Update the counter
            5. Release the lock

        Args:
            formatted_message [str]: The complete formatted log entry.

        Returns:
            None

        Notes:
            - Automatically handles log rotation
            - Falls back to terminal output on write failure
        """

        # Lock the thread to ensure thread safety
        with self.lock:
            # # Record the log write start time
            # start_time = time.ticks_us()

            # Check whether rotation is needed before writing the log
            if self.output != "terminal" and (self.log_count >= self.max_logs or self._should_rotate()):
                # Perform rotation
                self._rotate_log_file()

            # Write log; if file output is set, write to the file, otherwise output to the terminal
            if self.output == "terminal":
                # Output to terminal
                print(formatted_message)
            else:
                # If the file handle is valid, write to the log file
                if self.log_file:
                    # Write the message and append a newline
                    self.log_file.write(formatted_message + "\n")
                    # Flush the log message to the buffer immediately
                    self.log_file.flush()
                    # Increment the log count counter
                    self.log_count += 1

                else:
                    # If the file handle is invalid, output to the terminal
                    print("can not open file, output to terminal:", formatted_message)

            # # Record the log write end time
            # end_time = time.ticks_us()
            # # Calculate the log write duration
            # # Calculate the elapsed time
            # write_time = time.ticks_diff(end_time, start_time)
            # # Print the elapsed time in ms
            # print(f"write log cost {write_time / 1000} ms")

    def _rotate_log_file(self) -> None:
        """
        Perform log file rotation, including closing the current file, creating a new file, and resetting the counter.

        Args:
            None

        Returns:
            None

        Raises:
            OSError: Raised when a file operation fails
        """
        # # Debug output
        # print(f"[Rotation] Count:{self.log_count} Max:{self.max_logs}")

        # If the file handle is valid
        if self.log_file:
            # Close the current log file
            self.log_file.close()
        # Open a new log file
        self._open_log_file()
        # Reset the log count counter
        self.log_count = 0
        # Update the last rotation time
        self.last_rotation_time = time.time()

    @micropython.native
    def _should_rotate(self) -> bool:
        """
        Check whether the log rotation condition is met.

        Args:
            None

        Returns:
            bool: Returns True if rotation is needed, otherwise False
        """
        # Calculate the difference between the last rotation time and the current time
        current_hour = time.localtime()[3]
        last_hour = time.localtime(self.last_rotation_time)[3]
        time_diff = time.time() - self.last_rotation_time

        # The time condition is met if any of the following holds:
        # 1. The interval in hours has been reached
        # 2. The hour has changed and the time difference exceeds 1 hour (handles crossing days)
        return (time_diff >= self.rotate_interval * 3600) or \
            (current_hour != last_hour and time_diff >= 3600)

    @micropython.native
    def log(self, level: int, message: str) -> None:
        """
        Base logging method.

        Args:
            level (int): Log level (use LogLevel enum values)
            message (str): Log content to record

        Returns:
            None

        Raises:
            ValueError: Raised when the log level is invalid or message is empty.
        """
        # Check whether level is an integer and a value in LogLevel
        if not isinstance(level, int) or level not in [getattr(LogLevel, attr) for attr in dir(LogLevel) if
                                                       not attr.startswith('__')]:
            raise TypeError("level must be an integer of LogLevel enum")

        # Check whether message is a string or an empty string
        if not isinstance(message, str) or message == "":
            raise ValueError("message must be a string")

        # Record the log; first check the log level and filters
        if level >= self.level and self._apply_filters(level, message):
            formatted_message = self._format_message(level, message)
            # Output the log according to the settings
            self._write_log(formatted_message)

    def debug(self, message: str) -> None:
        """
        Record a DEBUG level log.

        Args:
            message (str): Debug information content

        Returns:
            None
        """
        # Record a debug log
        self.log(LogLevel.DEBUG, message)

    def info(self, message: str) -> None:
        """
        Record an INFO level log.

        Args:
            message (str): General information content

        Returns:
            None
        """
        # Record an info log
        self.log(LogLevel.INFO, message)

    def warning(self, message: str) -> None:
        """
        Record a WARNING level log.

        Args:
            message (str): Warning information content

        Returns:
            None
        """
        # Record a warning log
        self.log(LogLevel.WARNING, message)

    def error(self, message: str) -> None:
        """
        Record an ERROR level log.

        Args:
            message (str): Error information content

        Returns:
            None
        """
        # Record an error log
        self.log(LogLevel.ERROR, message)

    def critical(self, message: str) -> None:
        """
        Record a CRITICAL level log.

        Args:
            message (str): Critical error information content

        Returns:
            None
        """
        # Record a critical error log
        self.log(LogLevel.CRITICAL, message)

    def exception(self, exc: Exception, terminal_display: bool = True) -> None:
        """
        Record exception stack trace information.

        Args:
            exc (Exception): Exception object
            terminal_display (bool): Whether to display in the terminal, default is True

        Returns:
            None
        """
        # Print the exception stack trace to the terminal
        if isinstance(exc, Exception):
            # Also write the exception information to the log file
            if self.output != "terminal" and self.log_file:
                sys.print_exception(exc, self.log_file)

            # If the terminal display flag is True, print the exception information to the terminal
            if terminal_display:
                sys.print_exception(exc)

    def close(self) -> None:
        """
        Safely close the log file.

        Args:
            None

        Returns:
            None
        """
        # Close the log file if file output is used
        if self.output != "terminal" and self.log_file:
            self.log_file.close()

# ======================================== Initialization ==========================================

# ========================================  Main Program  ===========================================

3.2.1 Implementation Principle of the Logger Logging Class

In the aforementioned logger logging class, we provide the following methods:

Method Name

Description

__init__(self, name: str, level: int = LogLevel.DEBUG, output: str = "terminal", max_logs: int = 1000, rotate_interval: int = 24)

Initialize the logger, including the log name, level, output destination, maximum number of log entries and rotation interval.

add_filter(self, log_filter: Callable[[int, str], bool]) -> None

Add a log filter, which is a function that determines whether a given log entry will be recorded.

set_formatter(self, formatter: object) -> None

Set up the logger formatter, which is used to define the output format of logs.

log(self, level: int, message: str) -> None

To record a log entry, you need to specify the log level and the log content.

debug(self, message: str) -> None

Logs information at the DEBUG level.

info(self, message: str) -> None

Log information at the INFO level.

warning(self, message: str) -> None

Log information at the WARNING level.

error(self, message: str) -> None

Log information at the ERROR level.

critical(self, message: str) -> None

Log information at the CRITICAL level.

exception(self, exc: Exception, terminal_display: bool = True) -> None

Record exception information, and optionally choose whether to output the exception stack in the terminal.

close(self) -> None

Close the log file to ensure a safe exit.

_open_log_file(self) -> None

A private method that opens or creates a log file and initializes the counter.

_is_single_level_path(path: str) -> bool

Private method, used to check whether a path is a single-level folder path.

_apply_filters(self, level: int, message: str) -> bool

Private method that checks whether a log meets the criteria of all filters.

_format_message(self, level: int, message: str) -> str

A private method that formats log information into a human-readable string.

_write_log(self, formatted_message: str) -> None

A private method that writes logs to a file or outputs them to the terminal in a thread-safe manner.

_should_rotate(self) -> bool

A private method that checks whether the log rotation conditions are met.

_rotate_log_file(self) -> None

A private method that performs log file rotation, creates a new log file and closes the old one.

We have implemented methods for logging logs at different levels, such as debug, info, warning and so on, which mainly call the log method:

20 File System and Storage Architecture - Draft Document Tutorial
@micropython.native
def log(self, level: int, message: str) -> None:
    """
    Base logging method.

    Args:
        level (int): Log level (use LogLevel enum values)
        message (str): Log content to record

    Returns:
        None

    Raises:
        ValueError: Raised when the log level is invalid or message is empty.
    """
    # Check whether level is an integer and a value in LogLevel
    if not isinstance(level, int) or level not in [getattr(LogLevel, attr) for attr in dir(LogLevel) if
                                                   not attr.startswith('__')]:
        raise TypeError("level must be an integer of LogLevel enum")

    # Check whether message is a string or an empty string
    if not isinstance(message, str) or message == "":
        raise ValueError("message must be a string")

    # Record the log; first check the log level and filters
    if level >= self.level and self._apply_filters(level, message):
        formatted_message = self._format_message(level, message)
        # Output the log according to the settings
        self._write_log(formatted_message)

The log method first checks the log level and filters, then calls the _write_log method to output the formatted and filtered log information:

20 File System and Storage Architecture - Draft Document Tutorial
@micropython.native
def _write_log(self, formatted_message: str) -> None:
    """
    Thread-safe log write operation. Choose the write method according to the log output method; it can write to a file or the terminal.

    Execution flow:
        1. Acquire the thread lock
        2. Check the rotation condition
        3. Perform the actual write
        4. Update the counter
        5. Release the lock

    Args:
        formatted_message [str]: The complete formatted log entry.

    Returns:
        None

    Notes:
        - Automatically handles log rotation
        - Falls back to terminal output on write failure
    """

    # Lock the thread to ensure thread safety
    with self.lock:
        # # Record the log write start time
        # start_time = time.ticks_us()

        # Check whether rotation is needed before writing the log
        if self.output != "terminal" and (self.log_count >= self.max_logs or self._should_rotate()):
            # Perform rotation
            self._rotate_log_file()

        # Write log; if file output is set, write to the file, otherwise output to the terminal
        if self.output == "terminal":
            # Output to terminal
            print(formatted_message)
        else:
            # If the file handle is valid, write to the log file
            if self.log_file:
                # Write the message and append a newline
                self.log_file.write(formatted_message + "\n")
                # Flush the log message to the buffer immediately
                self.log_file.flush()
                # Increment the log count counter
                self.log_count += 1

            else:
                # If the file handle is invalid, output to the terminal
                print("can not open file, output to terminal:", formatted_message)

        # # Record the log write end time
        # end_time = time.ticks_us()
        # # Calculate the log write duration
        # # Calculate the elapsed time
        # write_time = time.ticks_diff(end_time, start_time)
        # # Print the elapsed time in ms
        # print(f"write log cost {write_time / 1000} ms")

_write_log () is mainly responsible for writing the formatted log messages to the terminal or a file:

20 File System and Storage Architecture - Draft Document Tutorial
  1. First, via the lock self.lock, we ensure that no race condition will occur when multiple threads write logs simultaneously, guaranteeing the atomicity of log writing, and then check whether the log rotation condition is met.
  2. And determine the log output address:
    • If the log output method is terminal (terminal), the log message will be printed directly.
    • If the log output method is set to file, the log messages will be written to the specified log file.
  3. If outputting to a file, after ensuring the file handle is valid, write the log message to the file; meanwhile, call the flush () method to immediately flush the buffer. Here, increment the log_count counter each time a log entry is written;

When checking whether the conditions for log rotation are met, the main check is whether the maximum number of log entries or the log rotation time interval has been reached; if the conditions are satisfied, call the _rotate_log_file () method to close the current log file, open a new log file (by calling the _open_log_file () method), and reset the log entry counter:

20 File System and Storage Architecture - Draft Document Tutorial
def _rotate_log_file(self) -> None:
    """
    Perform log file rotation, including closing the current file, creating a new file, and resetting the counter.

    Args:
        None

    Returns:
        None

    Raises:
        OSError: Raised when a file operation fails
    """
    # # Debug output
    # print(f"[Rotation] Count:{self.log_count} Max:{self.max_logs}")

    # If the file handle is valid
    if self.log_file:
        # Close the current log file
        self.log_file.close()
    # Open a new log file
    self._open_log_file()
    # Reset the log count counter
    self.log_count = 0
    # Update the last rotation time
    self.last_rotation_time = time.time()

@micropython.native
def _should_rotate(self) -> bool:
    """
    Check whether the log rotation condition is met.

    Args:
        None

    Returns:
        bool: Returns True if rotation is needed, otherwise False
    """
    # Calculate the difference between the last rotation time and the current time
    current_hour = time.localtime()[3]
    last_hour = time.localtime(self.last_rotation_time)[3]
    time_diff = time.time() - self.last_rotation_time

    # The time condition is met if any of the following holds:
    # 1. The interval in hours has been reached
    # 2. The hour has changed and the time difference exceeds 1 hour (handles crossing days)
    return (time_diff >= self.rotate_interval * 3600) or \
        (current_hour != last_hour and time_diff >= 3600)

When invoking the _open_log_file () method, if you select to output logs to a file, we will generate a new log file name based on the current time, open the file in append mode for logging, and the log file name will contain a timestamp:

@micropython.native
def _open_log_file(self) -> None:
    """
    Create/open the log file. If the file already exists, read its line count to initialize the counter; for a new file the counter starts from 0.

    Args:
        None

    Returns:
        None

    Raises:
        OSError: Raised when file creation fails

    Notes:
        - File name format: {name}_{year-month-day_hour-minute-second}.txt
        - Automatically falls back to terminal output on failure
    """
    # Get the current time: format is (year, month, mday, hour, minute, second, weekday, yearday)
    t = time.localtime()
    # Record the timestamp and format it as a year-month-day_hour-minute-second string (e.g., 20240917_143001)
    timestamp = "{:04}{:02}{:02}_{:02}{:02}{:02}".format(t[0], t[1], t[2], t[3], t[4], t[5])

    # If output is to the terminal
    if self.output == "terminal":
        # Set the log file handle to None; terminal mode needs no file operations
        self.log_file = None
        self.log_count = 0
    else:
        # Generate the log file name from the name and timestamp: name_timestamp.txt
        log_filename = f"{self.output}/{self.name}_{timestamp}.txt"

        # Try to create the directory (only single-level directories are handled)
        try:
            os.mkdir(self.output)
        except OSError:
            # Ignore if the directory already exists
            pass

        try:
            # Check whether the file already exists
            try:
                os.stat(log_filename)
                file_exists = True
            except OSError:
                file_exists = False

            # Open the file in append mode (created if it does not exist)
            self.log_file = open(log_filename, 'a')

            # Line count
            if file_exists:
                count = 0
                with open(log_filename, 'r') as f:
                    while True:
                        # Read in 128-byte chunks (suited to memory-constrained environments)
                        chunk = f.read(128)
                        # Reached the end of the file
                        if not chunk:
                            break
                        # Count the newline characters (one per line)
                        count += chunk.count('\n')
                self.log_count = count
            else:
                # If it is a newly created file, initialize the counter
                self.log_count = 0
        except OSError as e:
            # Failed to open the file; print the error message
            print(f"无法打开文件 {log_filename}: {e}")
            # Set the file handle to None
            self.log_file = None

_open_log_file () performs the following operations:

20 File System and Storage Architecture - Draft Document Tutorial
  1. Record the timestamp and generate a unique log file name: time.localtime () gets the current time and returns a tuple containing information such as year, month, day, hour, minute, second, etc.(year, month, mday, hour, minute, second, weekday, yearday), which is formatted via the format () method into the YYYYMMDD_HHMMSS format, for example: 20240917_143001. This timestamp is used to generate a unique log file name to avoid file name conflicts.
  2. Terminal output mode and log file mode:
    • If the user selects "terminal" as the log output method, no log file will be created, the log handle will be set to None, and the log counter will be set to 0 at the same time;
    • If the user passes in a log file retention path, then combine it with the log storage path self.output, log name self.name and timestamp timestamp to concatenate the complete log file name.
  3. Processing log files: First, check whether the file exists, then open it in append mode so that logs are appended to the end of the file, while counting the number of log lines and updating self.log_count. The purpose of counting the number of log lines here is to avoid duplicate numbering in the log rotation mechanism.
  4. Failed to open file for processing: If an OSError occurs (e. g., insufficient storage space, insufficient permissions), print an error message and downgrade to terminal mode (self.log_file = None).

In addition to using debug (), info (), warning (), error (), exception () and other methods to output common log messages, we can also record exception stack information, which is mainly implemented through the exception () method:

def exception(self, exc: Exception, terminal_display: bool = True) -> None:
    """
    Record exception stack trace information.

    Args:
        exc (Exception): Exception object
        terminal_display (bool): Whether to display in the terminal, default is True

    Returns:
        None
    """
    # Print the exception stack trace to the terminal
    if isinstance(exc, Exception):
        # Also write the exception information to the log file
        if self.output != "terminal" and self.log_file:
            sys.print_exception(exc, self.log_file)

        # If the terminal display flag is True, print the exception information to the terminal
        if terminal_display:
            sys.print_exception(exc)

This method will write the stack information of the exception object to the log file at the same time (if the log output mode is set to file and the file handle is valid), and decide whether to display the exception information on the terminal according to the settings;sys.print_exception () method is used to output the detailed exception information to the specified log file or terminal, which not only records the exception information, but also facilitates viewing and debugging on the terminal.

Here, we use a separate LogLevel class instead of defining the log level as a class variable of the Logger class:

20 File System and Storage Architecture - Draft Document Tutorial

This is primarily for the purposes of separation of duties and enhanced scalability:

3.2.2 Key Points for Implementing Log Filters and Log Formatters

To implement the complete logging function, we need the loggers, log filters and log formatters to work together, among which the log filters and log formatters need to be implemented by users themselves:

When creating log filters and log formatters, they need to meet specific interface requirements so that they can be properly integrated with the logging system:

3.2.2 Application Experiment of Logging Class

In the following code, a custom logging module is used to implement logging, filtering and rotation functions in the MicroPython environment; the logged content is controlled via log levels, length and keyword filters, and a color formatter is used to enhance log readability; the code also demonstrates how to handle and log exception stack information.

The following code can be found in the elegance-devkit v1\Demo\71 FileSys_Logger folder in the resource package we provide.

The sample code is shown below:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/16 2:40 PM
# @Author  : Li Qingshui
# @File    : main.py       
# @Description : File system experiment, using the logger module to implement logging

# ======================================== Import Modules =========================================

# Import the logging module
from logger import uLogLite, LogLevel
# Import the os module, used for file operations
import os
# Import time-related modules
import time
# Import hardware-related modules
import machine

# ======================================== Global Variables ============================================

# Define the file path for storing log data
logfile_path = "/logs"
# Create the log file directory
os.mkdir(logfile_path)

# ======================================== Functions ============================================

# Log level filter
def level_filter(level: int, message: str) -> bool:
    """
    Simple log filter that allows logs of INFO level and above to be recorded

    Args:
        level (int): Log level (use LogLevel enum values)
        message (str): Log message content

    Returns:
        bool: True means the log is recorded, False means it is not recorded
    """

    # Simple log filter that only allows logs of INFO level and above to be recorded
    return level >= LogLevel.INFO

# Log length filter
def length_filter(max_length: int) -> callable:
    """
    Generate a log length filter function

    Args:
        max_length (int): Maximum allowed log length (in bytes)

    Returns:
        callable[[int, str], bool]: The generated filter function
    """
    def filter_fn(level, message):
        return len(message) <= max_length
    return filter_fn

# Time period filter
def time_range_filter(start_time: str, end_time: str) -> callable:
    """
    Generate a time period filter function (24-hour format)

    Args:
        start_time (str): Start time (format 'HH:MM', e.g. '09:00')
        end_time (str): End time (format 'HH:MM', e.g. '18:00')

    Returns:
        callable[[int, str], bool]: The generated filter function

    Raises:
        ValueError: Raised when the time format is invalid
    """
    def filter_fn(level, message):
        # Get the hour and minute of the current time
        current_time = time.localtime()
        current_hour_minute = f"{current_time[3]:02}:{current_time[4]:02}"
        # Compare whether the current time is within the specified range
        return start_time <= current_hour_minute <= end_time
    return filter_fn

# Keyword filter
def keyword_filter(keywords: list) -> callable:
    """
    Generate a keyword filter function

    Args:
        keywords (list[str]): List of keywords that must be included

    Returns:
        callable[[int, str], bool]: The generated filter function
    """
    def filter_fn(level, message):
        # Check whether the log message contains any keyword
        return any(keyword in message for keyword in keywords)
    return filter_fn

# Print all .txt files in the directory
def print_txt_files_in_directory(directory: str) -> None:
    """
    Traverse and print the contents of all .txt files in the directory

    Args:
        directory (str): Path of the directory to traverse

    Returns:
        None

    Raises:
        OSError: Raised when directory access fails
    """
    try:
        # List all files and subdirectories in the directory
        files = os.listdir(directory)

        # Traverse all files in the directory
        for file in files:
            # Check whether the file ends with .txt
            if file.endswith('.txt'):
                # Print the file name
                file_path = directory + '/' + file
                print(f"Reading file: {file_path}")

                try:
                    # Open and read the file
                    with open(file_path, 'r') as f:
                        # Read the file content line by line and print it
                        for line in f:
                            # end='' prevents automatic newline
                            print(line, end='')
                except OSError as e:
                    print(f"Error reading file {file_path}: {e}")
                # Print an empty line to separate file contents
                print()
    except OSError as e:
        print(f"Error listing directory {directory}: {e}")

# ======================================== Custom Classes ============================================

# Terminal color formatter
class ColoredFormatter:
    """
    Terminal color formatter class that adds ANSI color control characters and a timestamp to log messages.

    This class automatically adds ANSI escape sequences of different colors to messages according to the log level, and adds a standardized timestamp.
    Supports color customization for all standard log levels (DEBUG/INFO/WARNING/ERROR/CRITICAL).

    Attributes:
        COLORS (dict): Color code mapping table, including:
            - LogLevel.DEBUG: blue
            - LogLevel.INFO: green
            - LogLevel.WARNING: yellow
            - LogLevel.ERROR: red
            - LogLevel.CRITICAL: red background
            - "ENDC": color reset code

    Methods:
        format(level, message):
            Core formatting method, adds color and timestamp
    """
    # Define color codes
    COLORS = {
        LogLevel.DEBUG: '\033[94m',     # Blue
        LogLevel.INFO: '\033[92m',      # Green
        LogLevel.WARNING: '\033[93m',   # Yellow
        LogLevel.ERROR: '\033[91m',     # Red
        LogLevel.CRITICAL: '\033[41m',  # Red background
        "ENDC": '\033[0m'               # Reset color
    }

    def format(self, level: int, message: str) -> str:
        """
        Format the log message, adding a timestamp and color control characters

        Args:
            level (int): Log level (use LogLevel enum values)
            message (str): Original log content

        Returns:
            str: Formatted log message, including:
                 - Color control characters (according to log level)
                 - Timestamp (YYYY-MM-DD HH:MM:SS format)
                 - Log level and original message

        Raises:
            ValueError: May be raised when the log level is invalid

        Notes:
            - Color control follows ANSI escape sequences
            - Timestamp uses UTC time
            - Unrecognized log levels use the default color
        """
        # Get the local time tuple of the current time
        local_time = time.localtime()

        # Build ISO 8601 format time string (YYYY-MM-DD HH:MM:SS)
        timestamp = "{:04}-{:02}-{:02} {:02}:{:02}:{:02}".format(
            local_time[0],  # Year (4 digits)
            local_time[1],  # Month (01-12)
            local_time[2],  # Day (01-31)
            local_time[3],  # Hour (00-23)
            local_time[4],  # Minute (00-59)
            local_time[5]  # Second (00-59)
        )

        # Get the color code for the corresponding level, default to ENDC (reset color)
        color = self.COLORS.get(level, self.COLORS["ENDC"])
        # Return the formatted message with color, reset color at the end
        return f"{color} [log]: {timestamp} - {level}: {message}{self.COLORS['ENDC']}"

# ======================================== Initialization ==========================================

# Power-on delay of 3 s
time.sleep(3)
# Print debug information
print("FreakStudio : Using logger module to save status of system")

# Create RTC object
rtc = machine.RTC()
# Set the RTC time to modify the system's internal time; time can also be obtained from an NTP server
# Set the system time to September 17, 2024, 14:40:00
rtc.datetime((2024, 9, 17, 0, 14, 40, 0, 0))
# Create a log object: log name is "MyLogger", log level is INFO, output to file, maximum number of logs is 20
# If the number of logs exceeds 20, a new file will be created
logger = uLogLite("MyLogger", LogLevel.INFO, logfile_path, max_logs=20)

# Add multiple filters
# Filter out logs below the INFO notice level
logger.add_filter(level_filter)
# Filter out logs longer than 50 characters
logger.add_filter(length_filter(50))
# Filter out logs that do not contain the 'FreakStudio' keyword
logger.add_filter(keyword_filter(["FreakStudio"]))
# Set to use the colored terminal formatter
logger.set_formatter(ColoredFormatter())

# ========================================  Main Program  ===========================================

# Test the logging module
print("Test logger module output log")
# Record debug information; this log will not be recorded because of the log level filter
logger.debug("FreakStudio : This is a debug message")
# Delay 30 ms
time.sleep_ms(30)

# Record an info log
logger.info("FreakStudio : This is an info message")
# This log will not be recorded because of the log length filter
logger.info("FreakStudio : This is a very very long long long long long info message,"
            "and the length is more than 50")
# Delay 30 ms
time.sleep_ms(30)

# Record a warning log
logger.warning("FreakStudio : This is a warning message")
# This log will not be recorded because of the log keyword filter
logger.warning("This is a warning message")
# Delay 30 ms
time.sleep_ms(30)

# Record an error log
logger.error("FreakStudio : This is an error message")
# Delay 30 ms
time.sleep_ms(30)

# Record a critical error log
logger.critical("FreakStudio : This is a critical message")
# Delay 30 ms
time.sleep_ms(30)

# Test the log rotation function
print("Test logger rotate log file")
# Record 100 logs in a loop to test the log rotation function
for i in range(20):
    # Record a debug log
    logger.debug(f"FreakStudio : This is a debug message {i}")
    time.sleep_ms(100)
    # Record an info log
    logger.info(f"FreakStudio : This is an info message {i}")
    time.sleep_ms(100)
    # Record a warning log
    logger.warning(f"FreakStudio : This is a warning message {i}")
    time.sleep_ms(100)
    # Record an error log
    logger.error(f"FreakStudio : This is an error message {i}")
    time.sleep_ms(100)
    # Record a critical error log
    logger.critical(f"FreakStudio : This is a critical message {i}")
    time.sleep_ms(100)

# Test the function of recording exception stack information in the log
print("Test logger record exception stack")
# Deliberately raise an exception
try:
    1 / 0
except Exception as e:
    # Record exception stack information and display it in the terminal at the same time
    logger.exception(e, terminal_display=True)

# Close the log file
logger.close()
# Output debug information
print("FreakStudio : logging file write done")

In the above code, we follow the process outlined below:

Here, we define a ColoredFormatter class to add colors to log messages for better readability in the terminal:

# Terminal color formatter
class ColoredFormatter:
    """
    Terminal color formatter class that adds ANSI color control characters and a timestamp to log messages.

    This class automatically adds ANSI escape sequences of different colors to messages according to the log level, and adds a standardized timestamp.
    Supports color customization for all standard log levels (DEBUG/INFO/WARNING/ERROR/CRITICAL).

    Attributes:
        COLORS (dict): Color code mapping table, including:
            - LogLevel.DEBUG: blue
            - LogLevel.INFO: green
            - LogLevel.WARNING: yellow
            - LogLevel.ERROR: red
            - LogLevel.CRITICAL: red background
            - "ENDC": color reset code

    Methods:
        format(level, message):
            Core formatting method, adds color and timestamp
    """
    # Define color codes
    COLORS = {
        LogLevel.DEBUG: '\033[94m',     # Blue
        LogLevel.INFO: '\033[92m',      # Green
        LogLevel.WARNING: '\033[93m',   # Yellow
        LogLevel.ERROR: '\033[91m',     # Red
        LogLevel.CRITICAL: '\033[41m',  # Red background
        "ENDC": '\033[0m'               # Reset color
    }

    def format(self, level: int, message: str) -> str:
        """
        Format the log message, adding a timestamp and color control characters

        Args:
            level (int): Log level (use LogLevel enum values)
            message (str): Original log content

        Returns:
            str: Formatted log message, including:
                 - Color control characters (according to log level)
                 - Timestamp (YYYY-MM-DD HH:MM:SS format)
                 - Log level and original message

        Raises:
            ValueError: May be raised when the log level is invalid

        Notes:
            - Color control follows ANSI escape sequences
            - Timestamp uses UTC time
            - Unrecognized log levels use the default color
        """
        # Get the local time tuple of the current time
        local_time = time.localtime()

        # Build ISO 8601 format time string (YYYY-MM-DD HH:MM:SS)
        timestamp = "{:04}-{:02}-{:02} {:02}:{:02}:{:02}".format(
            local_time[0],  # Year (4 digits)
            local_time[1],  # Month (01-12)
            local_time[2],  # Day (01-31)
            local_time[3],  # Hour (00-23)
            local_time[4],  # Minute (00-59)
            local_time[5]  # Second (00-59)
        )

        # Get the color code for the corresponding level, default to ENDC (reset color)
        color = self.COLORS.get(level, self.COLORS["ENDC"])
        # Return the formatted message with color, reset color at the end
        return f"{color} [log]: {timestamp} - {level}: {message}{self.COLORS['ENDC']}"

Among them, COLORS is a dictionary that defines the terminal color codes corresponding to different log levels:

Here, the different colors displayed in the terminal are achieved by using ANSI escape codes, which are the standard mechanism for controlling terminal text styles including effects like color, bold, underline, etc. : An ANSI escape code is a special character sequence that tells the terminal how to display text; it starts with \033 (or \e), followed by a square bracket[and a series of parameters, and ends with a letter.

20 File System and Storage Architecture - Draft Document Tutorial
The ANSI escape codes corresponding to different colors can be found at https://stackoverflow.com/questions/4842424/list-of-ansi-color-escape-sequences

In the format method for formatting log messages, we first obtain the current local time tuple, and then format the time tuple as a string in the format of YYYY-MM-DD HH: MM: SS , which represents the time of logging.

In the code for color selection and message construction:

    # Select color according to log level
    color = self.COLORS.get(level, self.COLORS["ENDC"])
    # Return the formatted message with color
    return f"{color} [log]: {timestamp} - {level}: {message}{self.COLORS['ENDC']}"

We retrieve the corresponding color code from the COLORS dictionary based on the log level; if the level is not defined in the dictionary, we use ENDC as the default value. We then construct the final log message, which includes the timestamp, log level, and log content, wrapped with the color code, and finally append ENDC at the end of the log message to reset the color.

Finally, we also define a function that is used to traverse all files with the. txt suffix in the specified directory and print the content of the files line by line:

# Print all .txt files in the directory
def print_txt_files_in_directory(directory: str) -> None:
    """
    Traverse and print the contents of all .txt files in the directory

    Args:
        directory (str): Path of the directory to traverse

    Returns:
        None

    Raises:
        OSError: Raised when directory access fails
    """
    try:
        # List all files and subdirectories in the directory
        files = os.listdir(directory)

        # Traverse all files in the directory
        for file in files:
            # Check whether the file ends with .txt
            if file.endswith('.txt'):
                # Print the file name
                file_path = directory + '/' + file
                print(f"Reading file: {file_path}")

                try:
                    # Open and read the file
                    with open(file_path, 'r') as f:
                        # Read the file content line by line and print it
                        for line in f:
                            # end='' prevents automatic newline
                            print(line, end='')
                except OSError as e:
                    print(f"Error reading file {file_path}: {e}")
                # Print an empty line to separate file contents
                print()
    except OSError as e:
        print(f"Error listing directory {directory}: {e}")

The function accepts a parameter directory, which indicates the directory to be traversed. First, we use os.listdir (directory) to list all files and subdirectories in the specified directory, then iterate over each entry in the files list, check the file suffix, process. txt files, and read the file content line by line.

The entire process of the system is shown in the following figure:

20 File System and Storage Architecture - Draft Document Tutorial

Burn the code, open the terminal, and the output is as follows:

20 File System and Storage Architecture - Draft Document Tutorial

As you can see, after the log file is fully written, we use the os.listdir (logfile_path) command to check the files in the current directory:

20 File System and Storage Architecture - Draft Document Tutorial

Display multiple files with timestamps in their names, then call the print_txt_files_in_directory function to view the file contents:

print_txt_files_in_directory(logfile_path)
20 File System and Storage Architecture - Draft Document Tutorial

It can be seen that the log filter functions properly, and none of the following logs have been recorded:

# Record debug information; this log will not be recorded because of the log level filter
logger.debug("FreakStudio : This is a debug message")
# This log will not be recorded because of the log length filter
logger.info("FreakStudio : This is a very very long long long long long info message,"
            "and the length is more than 50")
# This log will not be recorded because of the log keyword filter
logger.warning("This is a warning message")

Among the multiple logs recorded subsequently, they are stored in separate files:

20 File System and Storage Architecture - Draft Document Tutorial

As can be seen, the exception stack information is also recorded in the file:

20 File System and Storage Architecture - Draft Document Tutorial

3.2 Application Experiment.mp4 19.8 MB

It should be noted here that in the custom logger class, we write to the log file in append mode. As the file grows larger, the file system needs more time to locate the end of the file and perform the write operation. Therefore, in the logger class's _write_log method, we add code to calculate the time consumption:

@micropython.native
def _write_log(self, formatted_message: str) -> None:
    """
    Thread-safe log write operation. Choose the write method according to the log output method; it can write to a file or the terminal.

    Execution flow:
        1. Acquire the thread lock
        2. Check the rotation condition
        3. Perform the actual write
        4. Update the counter
        5. Release the lock

    Args:
        formatted_message [str]: The complete formatted log entry.

    Returns:
        None

    Notes:
        - Automatically handles log rotation
        - Falls back to terminal output on write failure
    """

    # Lock the thread to ensure thread safety
    with self.lock:
        # Record the log write start time
        start_time = time.ticks_us()
        
        ... ...
        
        # Record the log write end time
        end_time = time.ticks_us()
        # Calculate the time taken to write the log
        # Calculate elapsed time
        write_time = time.ticks_diff(end_time, start_time)
        # Print the elapsed time, in ms
        print(f"write log cost {write_time / 1000} ms")

Clear all files on the Raspberry Pi Pico, flash the code, and open the terminal:

20 File System and Storage Architecture - Draft Document Tutorial

It can be seen that the write time increases in stages, and the time consumption will also increase when a new file needs to be created.

Meanwhile, we can also use the os.stat ('/logs/MyLogger_20240917_144011.txt') command to check the file-related information:

20 File System and Storage Architecture - Draft Document Tutorial
>>> os.stat('/logs/MyLogger_20240917_144011.txt')
(32768, 0, 0, 0, 0, 0, 1690, 1726584014, 1726584014, 1726584014)

Here, the output information includes (mode, ino, dev, nlink, uid, gid, size, atime, mtime, ctime):

20 File System and Storage Architecture - Draft Document Tutorial

Here, apart from the filter functions and formatter classes we have implemented above, you can also define specific filter functions as needed, and please note the following at this point:

When customizing a formatter, it should be noted that apart from terminal formats with colors, the log formatter should support multiple output formats, and common formatting methods include:

Compared to the logging module of MicroPython, our custom logger class also provides core logging functions, including log levels, formatting, handlers (terminal and file), and exception handling; it also implements a thread-safe mechanism and supports log file rotation. However, the logging module of MicroPython offers more features and extensibility, such as more options for Filter and Handler, making it suitable for more complex logging requirements.

3.3 Implementation of the Simple Block Device Class

3.3.1 Implementation of the Abstract Base Class for Block Devices

As we mentioned earlier, MicroPython does not provide a built-in method for operating block devices, but it offers an abstract block device interface class. For our custom block device operation class to work properly, it needs to conform to the interfaces defined in the vfs.AbstractBlockDev abstract class, so as to enable file and storage device operations within the Virtual File System (VFS).

Below is a simple implementation example of a RAM block device (not recommended for practical use, for demonstration only):

class RAMBlockDev:
    def __init__(self, block_size, num_blocks):
        self.block_size = block_size
        self.data = bytearray(block_size * num_blocks)

    def readblocks(self, block_num, buf):
        for i in range(len(buf)):
            buf[i] = self.data[block_num * self.block_size + i]

    def writeblocks(self, block_num, buf):
        for i in range(len(buf)):
            self.data[block_num * self.block_size + i] = buf[i]

    def ioctl(self, op, arg):
        if op == 4:  # Get the number of blocks
            return len(self.data) // self.block_size
        if op == 5:  # Get the block size
            return self.block_size

Here, vfs.AbstractBlockDev is an abstract base class that defines the basic interfaces a block device should implement. Any custom block device class must implement these methods to be used for file system operations (such as littlefs, fatfs, etc.) via the VFS.While the RAMBlockDev class is a concrete implementation that inherits the interface of vfs.AbstractBlockDev, representing a block device that uses memory as the storage medium. It simulates the behavior of read and write operations on a block device, and provides the corresponding ioctl implementation for methods to obtain information such as the size and number of blocks of the block device.

In the following sample code, we first create a AbstractBlockDevInterface.py file to hold the abstract base class:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/18 PM 12:15
# @Author  : Li Qingshui
# @File    : AbstractBlockDevInterface.py.py       
# @Description : Defines the abstract base class AbstractBlockDev for block devices

# ======================================== Import Modules =========================================

# ======================================== Global Variables ============================================

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

# Custom abstract base class AbstractBlockDev, representing the basic operation interface of a block device
class AbstractBlockDev:
    """
    AbstractBlockDev class, defines a block device operation interface conforming to the MicroPython specification.

    This class provides an abstract interface for block devices, supporting a simple interface (block-aligned operations) and an extended interface (arbitrary-offset operations),
    and is compatible with the driver requirements of MicroPython filesystems (such as FAT and littlefs). Implementers must support at least
    ioctl(4) block count query; to support littlefs, the ioctl(6) block erase function must also be implemented.

    Attributes:
        (The following attributes are implemented by concrete subclasses)
        block_size (int): Block size (in bytes), usually exposed through ioctl(5)
        block_count (int): Total number of blocks, usually exposed through ioctl(4)

    Methods:
        __init__():
            Initialize the block device abstract base class
        readblocks(block_num, buf, offset=0):
            Read data blocks from the device into a buffer
        writeblocks(block_num, buf, offset=0):
            Write data to the device or erase blocks
        ioctl(op, arg):
            Control device operations and query parameters

    Note:
        - Simple interface: requires the buffer length to be aligned with the block size (offset=0)
        - Extended interface: supports arbitrary offset and length (offset≠0)
        - The following must be observed when writing:
            - The simple interface must handle erasing automatically
            - The extended interface prohibits implicit erasing
    """
    # Standard opcode constants (class attributes)
    IOCTL_INIT = 1      # Device initialization
    IOCTL_SHUTDOWN = 2  # Device shutdown
    IOCTL_SYNC = 3      # Data synchronization
    IOCTL_BLK_COUNT = 4 # Get block count (must be implemented)
    IOCTL_BLK_SIZE = 5  # Get block size (optional)
    IOCTL_BLK_ERASE = 6 # Block erase (required by littlefs)

    def __init__(self) -> None:
        """
        Initialize the block device abstract base class.

        Args:
            None

        Returns:
            None

        Note:
            - Must support at least ioctl(4) to get the block count
            - Implementing littlefs requires additional support for ioctl(6) block erase
        """
        pass

    def readblocks(self, block_num: int, buf: bytearray, offset: int = 0) -> None:
        """
        Read data blocks from the device into a buffer.

        Args:
            block_num (int): Starting block number (starting from 0).
            buf (bytearray): Buffer to store the read data.
            offset (int, optional): Byte offset within the block (default 0 means full block read).

        Returns:
            None

        Raises:
            OSError: If block_num is invalid (ENODEV).
            OSError: If the read fails (EIO).
            NotImplementedError: If the extended interface is not implemented (when offset≠0).
        """
        raise NotImplementedError("sub class must implement this method")

    def writeblocks(self, block_num: int, buf: bytearray | None, offset: int = 0) -> None:
        """
        Write data to the block device.

        Args:
            block_num (int): Target block number (starting from 0)
            buf (bytearray | None): Data to write (None means erase the block)
            offset (int, optional): Byte offset within the block (default 0 means full block write)

        Returns:
            None

        Raises:
            OSError: If block_num is invalid (ENODEV)
            OSError: If the write fails (EIO)
            OSError: If the device is read-only (EROFS)
            NotImplementedError: If the extended interface is not implemented (when offset≠0)
        """
        raise NotImplementedError("sub class must implement this method")

    def ioctl(self, op: int, arg: int) -> int | None:
        """
        Device control operation.

        Args:
            op (int): Opcode (class constants should be used):
                - `AbstractBlockDev.IOCTL_INIT` (1)       : Device initialization
                - `AbstractBlockDev.IOCTL_SHUTDOWN` (2)   : Device shutdown
                - `AbstractBlockDev.IOCTL_SYNC` (3)       : Data synchronization
                - `AbstractBlockDev.IOCTL_BLK_COUNT` (4)  : Get the total number of blocks of the device (must be implemented)
                - `AbstractBlockDev.IOCTL_BLK_SIZE` (5)   : Get the block size (optional)
                - `AbstractBlockDev.IOCTL_BLK_ERASE` (6)  : Erase block (required by littlefs)

            arg (int): Operation argument (depends on op); some operations need no argument:
                - `IOCTL_BLK_ERASE` (6) requires the block number to erase to be passed in
                - Other operations can pass `0` or `None`, as defined by the subclass

        Returns:
            int | None:
                - `IOCTL_BLK_COUNT` (4): Returns the total number of blocks of the device (int)
                - `IOCTL_BLK_SIZE` (5) : Returns the block size (int)
                - `IOCTL_BLK_ERASE` (6): Returns `0` on successful erase
                - Other operations return `None` by default

        Raises:
            NotImplementedError: If the subclass does not implement this method
        """
        raise NotImplementedError("sub class must implement this method")

# ======================================== Initialization ==========================================

# ========================================  Main Program  ===========================================

The purpose of this type of design is to unify the operation methods of different storage media (such as Flash, RAM, SD card, etc.), support the underlying driver requirements of file systems including FAT and littlefs, and provide both simple interfaces (block-aligned) and extended interfaces (arbitrary offset).

Here, we follow the method specified in the official documentation to define the vfs.AbstractBlockDev class:

20 File System and Storage Architecture - Draft Document Tutorial

When reading block device data via the readblocks method and writing block device data via the writeblocks method, the offset offset is set as a default parameter to support non-block-aligned reads and writes of arbitrary-length data; additionally, for the device control operation ioctl method, standard operation code constants are defined in the class attributes:

# Standard opcode constants (class attributes)
IOCTL_INIT = 1      # Device initialization
IOCTL_SHUTDOWN = 2  # Device shutdown
IOCTL_SYNC = 3      # Data synchronization
IOCTL_BLK_COUNT = 4 # Get block count (must be implemented)
IOCTL_BLK_SIZE = 5  # Get block size (optional)
IOCTL_BLK_ERASE = 6 # Block erase (required by littlefs)

3.3.2 Implementation of RAM Block Device Class

Specific block device classes (such as RAM block devices, SD cards, etc.) need to inherit from the abstract base class vfs.AbstractBlockDev and implement the methods defined within it. We have implemented a simple RAM block device RAMBlockDev, and the RAMBlockDev class is placed in a separate. py file:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/18 PM 12:15
# @Author  : Li Qingshui
# @File    : ram_block_dev.py.py       
# @Description : Inherits the AbstractBlockDev class and implements a simple RAM block device RAMBlockDev, including block read/write operations and device control functions

# ======================================== Import Modules =========================================

# Import the abstract base class of the virtual file block device
from AbstractBlockDevInterface import AbstractBlockDev

# ======================================== Global Variables ============================================

# Define error code constants, conforming to POSIX standard error codes
ENOERR = 0       # Operation successful
EPERM = 1        # Operation not permitted
EIO = 5          # I/O error
ENODEV = 19      # Invalid device/block number
EROFS = 30       # Read-only file system
EINVAL = 22      # Invalid argument

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

class RAMBlockDev(AbstractBlockDev):
    """
    RAM-simulated block device implementation, using an in-memory byte array as the storage medium

    Attributes:
        block_size (int): Size of each block in bytes (read-only)
        num_blocks (int): Total number of blocks on the device (read-only)
        initialized (bool): Device initialization state flag
    """

    def __init__(self, block_size: int, num_blocks: int) -> None:
        """
        Initialize the RAM block device

        Args:
            block_size (int): Size of each block in bytes (must be ≥512)
            num_blocks (int): Total number of blocks (must be ≥1)

        Raises:
            ValueError: Raised when block_size<512 or num_blocks<1
        """
        # Check parameters
        if block_size < 512:
            raise ValueError("Block size must be ≥512 bytes")
        if num_blocks < 1:
            raise ValueError("At least 1 block required")

        # Block size
        self.block_size = block_size
        # Number of blocks
        self.num_blocks = num_blocks
        # Create a byte array as the device storage
        self.data = bytearray(block_size * num_blocks)
        # Device initialization state flag
        self.initialized = False

    def readblocks(self, block_num: int, buf: bytearray, offset: int = 0) -> None:
        """
        Read data from the device (supports extended interface)

        Args:
            block_num (int): Starting block number (0 ≤ block_num < num_blocks)
            buf (bytearray): Buffer that receives the data
            offset (int): Byte offset within the block (default 0)

        Returns:
            None

        Raises:
            OSError: If block_num is out of range (uses ENODEV).
            OSError: If the read exceeds the current block range (uses EIO).
        """
        # Check whether block_num is within the valid range
        if not 0 <= block_num < self.num_blocks:
            raise OSError(ENODEV)

        # Calculate the start position of the read
        start = block_num * self.block_size + offset
        # Calculate the end position of the read
        end = start + len(buf)

        # Make sure the data read does not exceed the block range
        if end > (block_num + 1) * self.block_size:
            raise OSError(EIO)

        # Read data from the device storage into the buffer
        buf[:] = self.data[start:end]

    def writeblocks(self, block_num: int, buf: bytearray | None, offset: int = 0) -> None:
        """
        Write data to the device (supports erase operation)

        Args:
            block_num (int): Target block number (0 ≤ block_num < num_blocks)
            buf (bytearray|None):
                - bytearray: Data to write
                - None: Erase the specified block (offset must be 0)
            offset (int): Byte offset within the block (default 0)

        Returns:
            None

        Raises:
            OSError: If block_num is out of range (uses ENODEV).
            OSError: If the write exceeds the current block range (uses EIO).
            OSError: If offset ≠ 0 during an erase operation (uses OSError(22) for invalid argument).
        """
        # Check whether block_num is within the valid range
        if not 0 <= block_num < self.num_blocks:
            raise OSError(ENODEV)

        # Perform block erase operation
        if buf is None:
            if offset != 0:
                # Invalid argument, erase must be done on a whole block
                raise OSError(EINVAL)
            # Erase the specified block
            start = block_num * self.block_size
            end = (block_num + 1) * self.block_size
            self.data[start:end] = bytes(self.block_size)
            return

        # Calculate the start position of the write
        start = block_num * self.block_size
        # Calculate the end position of the write
        end = start + len(buf)

        # Make sure the data written does not exceed the block range
        if end > (block_num + 1) * self.block_size:
            raise OSError(EIO)

        # Write data to the device storage
        self.data[start:end] = buf

    def ioctl(self, op: int, arg: int | None = None) -> int | None:
        """
        Device control operation.

        Args:
            op (int): Opcode (class constants should be used):
                - AbstractBlockDev.IOCTL_INIT (1)       : Initialize the device.
                - AbstractBlockDev.IOCTL_SHUTDOWN (2)   : Shut down the device.
                - AbstractBlockDev.IOCTL_SYNC (3)       : Synchronize data.
                - AbstractBlockDev.IOCTL_BLK_COUNT (4)  : Get the total number of blocks on the device (must be implemented).
                - AbstractBlockDev.IOCTL_BLK_SIZE (5)   : Get the block size (optional).
                - AbstractBlockDev.IOCTL_BLK_ERASE (6)  : Erase a block (required by littlefs).
            arg (int | None, optional): Operation argument, depends on op:
                - For IOCTL_BLK_ERASE (6): pass the block number to erase (int).
                - For other operations: can be None or 0, as defined by the subclass.

        Returns:
            int | None:
                - For IOCTL_BLK_COUNT (4): returns the total number of blocks on the device (int).
                - For IOCTL_BLK_SIZE (5) : returns the block size (int).
                - For IOCTL_BLK_ERASE (6): returns 0 if the erase succeeds.
                - For other operations: returns 0 or None by default.

        Raises:
            OSError: If the block number is out of range during erase (uses ENODEV).
            OSError: If op is not supported (uses EPERM).
        """
        # Handle different operations
        # Initialize the device
        if op == AbstractBlockDev.IOCTL_INIT:
            self.initialized = True
            # Return 0 to indicate initialization succeeded
            return 0
        # Shut down the device
        elif op == AbstractBlockDev.IOCTL_SHUTDOWN:
            self.initialized = False
            return 0
        # Synchronize the device
        elif op == AbstractBlockDev.IOCTL_SYNC:
            # In a RAM block device, writes take effect immediately, so no extra synchronization is needed
            # Return 0 to indicate synchronization succeeded
            return 0
        # Get the number of blocks
        elif op == AbstractBlockDev.IOCTL_BLK_COUNT:
            # Return the total number of blocks
            return self.num_blocks
        # Get the block size
        elif op == AbstractBlockDev.IOCTL_BLK_SIZE:
            # Return the block size
            return self.block_size
        # Erase a block
        elif op == AbstractBlockDev.IOCTL_BLK_ERASE:
            # Check whether the argument was passed correctly
            if arg is None:
                raise OSError(ENODEV)
            # Get the block number to erase
            block_num = arg
            if 0 <= block_num < self.num_blocks:
                # Calculate the start position of the erase
                start = block_num * self.block_size
                # Calculate the end position of the erase
                end = (block_num+1)*self.block_size
                # Set the data of the specified block to all 0 (erase)
                self.data[start:end] = bytes(self.block_size)
                # Return 0 to indicate erase succeeded
                return 0
            else:
                # Block number out of range
                raise OSError(ENODEV)
        # Handle unsupported operation codes
        else:
            # Raise an exception
            raise OSError(EPERM)

# ======================================== Initialization ==========================================

# ========================================  Main Program  ===========================================

RAMBlockDev class emulates a RAM-based block device, which uses an in-memory bytearray to store data:

In the initialization method, organize multiple blocks into a contiguous byte array:

self.data = bytearray(block_size * num_blocks)

Then a block addressing formula can be derived, namely start address of block N = N * block_size.

In the read operation (readblocks), the following operations are performed in sequence:

  1. Calculate the starting position for reading: start = block_num * self.block_size + offset.
  2. Calculate the end position for reading: end = start + len (buf).
  3. Boundary Check:
    • block_num is out of range (throws OSError (ENODEV)).
    • end must not exceed the block size of block_num; otherwise, OSError (EIO) will be thrown.
  4. Data Copy: buf[:] = self.data[start: end]: Assigns a slice of a bytearray to buf to avoid extra memory allocation by Python.

In a write operation (writeblocks), the process is similar to that of a read operation (readblocks): first, boundary checking is performed, then the start and end ranges of the write address are calculated. During writing, the bytearray is directly modified to avoid creating additional objects. Additionally, when buf= None, an erase operation is executed (filling the entire block with 0x00).

In the control command ioctl (), corresponding operations are performed according to the input device management instruction and entry parameters:

20 File System and Storage Architecture - Draft Document Tutorial

In addition, we have defined error codes (errno) compliant with the POSIX standard (Portable Operating System Interface) to indicate the reasons for the failure of system calls or library function execution:

We can learn more about it via the following link:

20 File System and Storage Architecture - Draft Document Tutorial

https://docs.particle.io/reference/device-os/api/debugging/posix-errors/

3.3.3 Application Experiment

The following code can be found in the elegance-devkit v1\Demo\72 FileSys_BlockDev folder in the resource package we provide.

This block device can be used to create and mount a file system, as well as to create, read from and write to files. In the following code, we test the RAM-based block device RAMBlockDev class, including the verification of write, read and control operations. The sample code is as follows:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/18 PM 12:15
# @Author  : Li Qingshui
# @File    : main.py       
# @Description : Test the functionality of the RAMBlockDev class, including verification of write, read, and control operations

# ======================================== Import Modules =========================================

# Import the custom block device implementation class
from ram_block_dev import RAMBlockDev
# Import the virtual file class
import vfs
# Import time-related modules
import time

# ======================================== Global Variables ============================================

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

# ======================================== Initialization ==========================================

# Power-on delay of 3 s
time.sleep(3)
# Print debug information
print("FreakStudio : RAMBlockDev Write and Read Test")

# Create a RAM block device with a block size of 512 bytes and 50 blocks in total
bdev = RAMBlockDev(512, 50)
# Create a FAT file system on the block device
vfs.VfsFat.mkfs(bdev)
# Mount the block device to the /ramdisk directory of the virtual file system
vfs.mount(bdev, '/ramdisk')

# ========================================  Main Program  ===========================================

# In the created virtual file system, create a file hello.txt and open it
with open('/ramdisk/hello.txt', 'w') as f:
    # Write content
    f.write('FreakStudio : Hello World')

# Read the file content and print it
print(open('/ramdisk/hello.txt').read())

In this code, we first create a RAM block device with a block size of 512 bytes and a total of 50 blocks, create a FAT file system on the block device, and mount the block device to the /ramdisk directory so that it can be used as a file system; then in the mounted virtual file system /ramdisk, we create and open a file named hello.txt, write data into it and then read the data back.

Burn the code, open the terminal, and the following content will be displayed:

20 File System and Storage Architecture - Draft Document Tutorial

It can be seen that the file system and files have been created successfully, and the data written to the file has been read successfully.

We can try to read the block data of the RAM block device bdev. First, create a bytearray buffer with a size of 512 bytes:

buf = bytearray(512)

Then use the bdev readblocks method to read block data. First, read the 0th block of data, which is the boot sector data:

>>> bdev.readblocks(0, buf, 0)
>>> buf
bytearray(b'\xeb\xfe\x90MSDOS5.0\x00\x02\x01\x01\x00\x01\x00\x022\x00\xf8\x01\x00?\x00\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x00).\x00!RNO NAME    FAT     \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00U\xaa')

The output data includes the boot sector signature, OEM identifier, key BPB parameters, etc. :

20 File System and Storage Architecture - Draft Document Tutorial
\x00\x02   # Bytes per sector (usually 512, i.e. 0x0200)
\x01       # Sectors per cluster (1 sector)
\x01\x00   # Number of reserved sectors (usually 1)
\x01       # Number of FAT tables (usually 2)
\x00\x02   # Number of root directory entries (32 root directory entries)
2\x00\xf8  # Media descriptor (0xF8 means hard disk)

It also includes information such as the number of hidden sectors and the number of sectors per FAT table.

Then read the first block of data, which is the FAT table data:

>>> bdev.readblocks(1, buf, 0)
>>> buf
bytearray(b'\xf8\xff\xff\xff\x0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')

The output data content includes the information of reserved clusters and file cluster chains:

20 File System and Storage Architecture - Draft Document Tutorial

Then we read the second block of data, which here corresponds to the FAT directory area:

>>> bdev.readblocks(2, buf, 0)
>>> buf
bytearray(b'HELLO   TXT \x18\x00.\x00!R\x00\x00\x00\x00.\x00!R\x02\x00\x19\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')

The output data includes the directory information of the files that have been stored so far:

20 File System and Storage Architecture - Draft Document Tutorial

We can try to find the data blocks storing the content of the hello.txt file with the following code:

search_str = b'FreakStudio : Hello World'

for block_num in range(50):  # Scan all 50 blocks
    buf = bytearray(512)
    bdev.readblocks(block_num, buf, 0)
    if search_str in buf:
        print(f"Found file content in block: {block_num}")
        break

The output result is 34. Let's check the content in the 34th block:

Found file content in block: 34
>>> bdev.readblocks(34, buf, 0)
>>> buf
bytearray(b'FreakStudio : Hello World\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')

It can be seen that the verification is correct, hello.txt and the content of the file is stored in the 34th data block.

Actually, when formatting a file system, the main process can be summarized as follows:

20 File System and Storage Architecture - Draft Document Tutorial
  1. Call bdev.ioctl (4) to obtain the number of blocks;
  2. Call bdev.ioctl (5) to get the block size;
  3. Write FAT file system metadata to the storage medium, including the boot sector, FAT table, root directory area, etc.

When writing to a file, perform the following operations:

20 File System and Storage Architecture - Draft Document Tutorial
  1. The file system searches for free clusters (by calling bdev.readblocks () to read the FAT table);
  2. Allocate cluster chains and update the FAT table (call bdev.writeblocks ());
  3. Write the content to the file.

When reading a file, perform the following operations:

20 File System and Storage Architecture - Draft Document Tutorial
  1. Locate the starting cluster of the file via the directory entry.
  2. Call bdev.readblocks () to read file data.
  3. Returns the content to the application layer.

3.4 SD Card Mounting File System

3.4.1 Basic Knowledge of SD Card

Anatomy of SD Card Communication Protocol: Structure of CMD Commands and Their Response Messages

3.4.1.1 Introduction and Classification of SD Cards

An SD card is a removable storage device widely used in digital devices, featuring small size, low power consumption and large capacity. It is mainly used to store information such as data, music, videos and images, and is widely applied in mobile phones, cameras, tablets, embedded devices and other products.

20 File System and Storage Architecture - Draft Document Tutorial

Based on their physical dimensions, SD cards can be categorized into the following types:

20 File System and Storage Architecture - Draft Document Tutorial

Depending on their storage capacity and supported protocols, SD cards can be categorized into the following types:

20 File System and Storage Architecture - Draft Document Tutorial

It should be noted here that different file systems support different maximum capacity limits and applicable devices:

20 File System and Storage Architecture - Draft Document Tutorial

Here, let’s supplement two basic concepts:

Wherein:

The "Speed Class" of an SD card guarantees its minimum write speed (a sufficiently fast speed prevents it from being laggy). Corresponding to different usage scenarios, there are mainly the following common speed classes:

3.4.1.2 Basic Structure and Working Principle of SD Card

The mechanical structure of an SD card consists of a plastic casing, metal pins, a memory chip, and a controller. Its casing is typically made of sturdy plastic material, which can withstand physical impacts and environmental factors:

20 File System and Storage Architecture - Draft Document Tutorial

An SD card is like a "USB flash drive with a smart butler", and it functions through the following units:

The bottom of an SD card features multiple metal pins for data transmission, power supply and signal control; a standard SD card has 9 pins, while a microSD card has 8:

20 File System and Storage Architecture - Draft Document Tutorial

The 9 pins of an SD card can be divided into four data transfer pins DAT0 to DAT3, a command pin CMD, ground pins VSS and power supply pins VDD, and most SD cards also have a write protect (WP) switch in standard SD cards:

20 File System and Storage Architecture - Draft Document Tutorial

Before introducing the functions of each pin, we need to know that SD cards have two common operation modes:

The following are the corresponding functions of each pin in the two modes:

Pin numberSD Mode (high-speed mode)SPI Mode (commonly used in embedded systems)
1Name: CD/DAT3
Type: I/O/PP
Function: card insertion detection + data transfer pin (bit 3)
Name: CS
Type: I
Function: chip select (the switch with which an external device "selects" the SD card, active low)
2Name: CMD
Type: I/O/PP
Function: send/receive commands and responses (e.g., the "read data" command)
Name: DI
Type: I
Function: data input (the SD card receives data from the external device)
3Name: VSS1
Type: S
Function: power ground (connect to the development board's GND)
Name: VSS
Type: S
Function: power ground (same as left)
4Name: VDD
Type: S
Function: power supply (connect to 3.3V, do not connect to 5V! It will burn out the card)
Name: VDD
Type: S
Function: power supply (same as left)
5Name: CLK
Type: I
Function: clock signal (the "metronome" for synchronous communication)
Name: SCLK
Type: I Function:
Clock signal (same as left)
6Name: VSS2
Type: S
Function: power ground (auxiliary ground, also connect to GND)
Name: VSS2
Type: S
Function: power ground (same as left)
7Name: DAT0
Type: I/O/PP
Function: data transfer pin (bit 0)
Name: DO
Type: O/PP
Function: data output (the SD card sends data to the external device)
8Name: DAT1
Type: I/O/PP
Function: data transfer pin (bit 1)
Name: RSV
Function: reserved (leave unconnected)
9Name: DAT2
Type: I/O/PP
Function: data transfer pin (bit 2)
Name: RSV
Function: reserved (leave unconnected)

The abbreviations of pin types in the above table are as follows:

There is a hardware switch on the side of the SD card, and meanwhile the inside of the card will detect the level of a specific pin (usually pin 8):

In addition to pins, controllers, and NAND flash memory, there are also multiple registers inside the SD card that manage the card's basic information, status, configuration and operating conditions, as shown in the following figure:

20 File System and Storage Architecture - Draft Document Tutorial

The internal structure of an SD card features a "hierarchical collaboration" design, with each component having clearly defined functions:

Below are some key registers and their functions:

Register nameSimplified explanationCore function
CIDCard "ID number"Stores the SD card's unique identifier (different for every card, like a person's ID number)
RCACard "temporary address"A "temporary number" generated when the SD card is initialized (lets the external device find it quickly; not used in SPI mode)
DSRCard "output configuration table"Configures the SD card's signal output strength (optional feature, beginners need not pay much attention)
CSDCard "capability specification"Stores the SD card's operating conditions and storage capabilities (e.g., maximum capacity, read/write speed limits)
SCRCard "special function table"Stores the SD card's special features (e.g., whether high-speed mode is supported)
OCRCard "operating conditions table"Stores the SD card's operating voltage range and whether it is currently ready (e.g., whether it works normally after power-up)
SSRCard "proprietary feature table"Stores the SD card's unique feature information (many details, generally not needed)
CSRCard's "current status table"Stores the SD card's real-time status (e.g., whether it is currently "read/write" or "read-only")

The collaboration process between external devices (such as microcontrollers) and SD cards is as follows:

  1. External devices send commands to the SD card controller via "pins"(such as "read your identification number");
  2. The controller retrieves information from the corresponding register (e. g., CID) and transmits it back to the external device via the pins;
  3. When reading or writing data, the controller manages the "cells" of the NAND flash (allocating space, storing/retrieving data) in accordance with instructions;
  4. During this process, the controller employs "wear leveling"(storing data in different blocks in rotation to prevent any single block from wearing out prematurely) and "error correction"(automatically fixing minor errors in data) to protect the SD card and the stored data.

3.4.1.3 Operating Modes of SD Card

You can analogize the interaction between a host device (such as a development board or a mobile phone) and an SD card to "a mobile phone connecting to a new USB flash drive":

The operating modes of an SD card mainly consist of two phases: the Card Identification Mode and the Data Transfer Mode. In the Card Identification Mode, the host aims to identify and initialize the SD card to get it ready for entering the Data Transfer Mode, which is the actual phase for reading and writing data:

20 File System and Storage Architecture - Draft Document Tutorial

The "Idle/Ready/Ident" in the flowchart refers to the status (i. e., the current phase) of the SD card, while "CMD0/CMD8" are the commands sent by the host to the SD card. The entire process is as follows:

  1. Initial State: Idle
    • When the SD card is inserted into the host (or powered on), the host will send "CMD0 (reset command)" — which is equivalent to the host saying: "Hello, reset yourself and get ready to start communication!";
    • At this point, the SD card will enter the "idle state", select the operation mode (SPI mode or SD bus mode) at the same time, and wait for the next command from the host.
  2. Check voltage: Send CMD8
    • The host sends "CMD8" — which is equivalent to asking: "Do you support the voltage I'm using here? ";
    • If the SD card does not support this voltage, it will return to the idle state; if it does, it will proceed to the next step.
  3. Inquiry preparation: Send ACMD41
    • The host sends "ACMD41" — which is equivalent to asking: "Are you ready? ";
    • If the SD card is ready, it will enter the "Ready State"; if it is not ready, it will return a "busy" signal, and the host will wait for a while before querying again.
  4. Need "ID card": Send CMD2
    • The host sends "CMD2" — which is equivalent to saying: "Send me your unique identifier (the CID register, just like an ID number)!";
    • The SD card enters the "Identification State" and transmits its own CID (information such as manufacturer and serial number) to the host.
  5. Assign a "temporary number": send CMD3
    • The host sends "CMD3" — which is equivalent to saying: "Here’s a temporary number (RCA, like a parcel pickup code) for you, I’ll refer to you by this later!";
    • The SD card enters the "Stand-by State", and obtains this RCA to facilitate the host to quickly locate it subsequently.
  6. Selected card: Send CMD7 to enter data transfer mode
    • The host sends "CMD7" — which is equivalent to saying: "I'm selecting you, get ready to start data transmission!";
    • At this point, the SD card switches from "Card Identification Mode" to "Data Transfer Mode", and file read/write operations can commence.

In addition, there are some special procedures:

20 File System and Storage Architecture - Draft Document Tutorial
3.4.1.4 Communication Protocol and Command Response Model of SD Card

The Communication Protocol of the SD card mainly includes the SD bus protocol and the SPI protocol:

SD bus protocolUses a CMD line + 4 data lines (DAT0~DAT3), with a CLK clock line, a VDD power line, and a VSS ground lineSupports parallel data transfer; bus width is configurable as 1-bit or 4-bitCommands are transmitted in a 48-bit frame structure with check bits, supporting normal commands (CMD) and application-specific commands (ACMD)Scenarios with high transfer rate requirements, such as consumer digital devices (cameras, phones)
SPI protocolUses 4 core pins (CLK clock line, MOSI master-out data, MISO slave-out data, CS chip select line)Serial data transfer, supports half-duplex communication onlyCommands use a single byte as the basic frame structure, and data is transferred as a continuous byte stream; the protocol is simplified, with no complex multi-line configurationEmbedded development scenarios with low hardware interface requirements, easy to implement through the MCU's SPI peripheral
3.4.1.4.1 Command Response Model in SD Bus Protocol

SD card communication adopts a command-response mode, where the host sends commands to the SD card to initiate specific operations, and the SD card responds to the received commands; the commands CMD and responses RESPONSE are serially transmitted via the CMD line, while data DATA is transmitted via the data lines (DAT lines), ensuring that command and data transmission do not interfere with each other.

20 File System and Storage Architecture - Draft Document Tutorial

Instruction CMD is formatted as follows, including a start bit, transmission bits, command content (command and address information or parameters), and a stop bit:

20 File System and Storage Architecture - Draft Document Tutorial

The command frame of the SD bus protocol is of a fixed 48-bit length, and its frame structure is defined sequentially from the most significant bit to the least significant bit as follows:

SD card commands are divided into two categories: standard commands (CMD) and application-specific commands (ACMD), which have distinct execution logics:

The commonly used general commands are as follows:

Command codeCommand nameCore function
CMD0GO_IDLE_STATEResets the SD card and puts it into the idle state, used to start the initialization process
CMD2ALL_SEND_CIDInstructs the SD card to send its own CID register data (unique card identifier)
CMD3SEND_RELATIVE_ADDRThe host assigns a relative address (RCA) to the SD card; the SD card returns this address and enters the standby state
CMD7SELECT/DESELECT_CARDSelects or deselects the SD card by RCA; once selected, the card enters data transfer mode
CMD17READ_SINGLE_BLOCKInstructs the SD card to read a single data block; the target block address parameter is required
CMD18READ_MULTIPLE_BLOCKInstructs the SD card to read multiple data blocks continuously; CMD12 must be sent to stop the transfer
CMD24WRITE_BLOCKInstructs the SD card to write a single data block; the target block address parameter is required
CMD25WRITE_MULTIPLE_BLOCKInstructs the SD card to write multiple data blocks continuously; CMD12 must be sent to stop the transfer
CMD55APP_CMDNotifies the SD card that an application-specific command (ACMD) will follow, providing the prerequisite for ACMD execution

The following is a list of common ACMD commands in the SD card protocol:

20 File System and Storage Architecture - Draft Document Tutorial

After receiving a valid command, the SD card must return a response frame via the CMD line. Response frames are categorized into 4 standard types by function and length, with four encoding schemes depending on content, and have a length of either 48 bits or 136 bits:

20 File System and Storage Architecture - Draft Document Tutorial

Here, we summarize the common response types of SD cards:

Response typeLengthKey features and uses
R148The most commonly used response type; contains the SD card's current status bits (such as idle, ready, error states, etc.) and is used for the responses to most CMD commands (such as CMD0, CMD3, CMD55)
R2136Long response frame, used to return SD card register data, such as the responses to CMD2 (returns CID) and CMD9 (returns CSD)
R348Used to return OCR register data, corresponds only to the CMD1/ACMD41 commands, and carries key information such as the card's operating voltage range and capacity type flag
R448Used for specific command responses in SPI protocol mode, carrying the SD card's status and configuration parameters

It should be noted that all response frames contain CRC check bits, and the host can only determine that the response is valid after verifying that the CRC is correct.

Data transmission of the SD card is based on the block transfer mechanism, the data frame format is related to the bus width, and it supports two modes: single-block transmission and multi-block transmission.

20 File System and Storage Architecture - Draft Document Tutorial

Wherein:

Data frame transmission is divided into two modes: normal data transmission and wide data transmission, which correspond to different bus widths.

20 File System and Storage Architecture - Draft Document Tutorial
3.4.1.4.2 Command Response Model in SPI Protocol

Similar to the SD bus protocol, the SPI protocol also consists of commands, responses, and data block markers. All communication between the host and the card is controlled by the host (master device), which initiates each transmission transaction by pulling the CS signal low. The data blocks in SPI can be as large as a write block or as small as a single byte; for SDHC and SDXC cards, the block length is fixed at 512 bytes. It should be noted that the write protection commands (CMD28, CMD29, and CMD30) are not supported in SPI mode.

20 File System and Storage Architecture - Draft Document Tutorial

In SPI mode, the card initialization process is as follows:

  1. Power-on and Mode Selection: After power-on, the SD card enters Idle State, and the host sends CMD0 (reset command):
    • If the CS signal is pulled low (CS Asserted), the SD card enters the SPI Operation Mode;
    • Otherwise, enter the SD bus mode.
  2. Voltage Compatibility Check: Hosts complying with the Version 2.00 specification shall send CMD8:
    • If the SD card returns "Non supported voltage range", the host will stop accessing it;
    • If voltage support is available, send CMD58 to read the OCR register and obtain the voltage range supported by the SD card.
  3. SD Card Initialization: Host sends ACMD41 (CMD55 must be sent first):
    • If the SD card returns "busy"(shown as "card is busy" in the diagram), it indicates that the card is undergoing internal initialization, and the host needs to wait until the card is ready.
    • After initialization is complete, the host can CMD58 read the OCR again to confirm the card capacity information (CCS identifier).
  4. Validity Check: If the SD card does not respond to CMD8 / ACMD41, it will be determined as "Not SD Memory Card", and the host will stop the operation.

In SPI mode, the SD card supports Single Block Read and Multiple Block Read operations, which correspond respectively to CMD17 and CMD18 commands in the SD card protocol:

20 File System and Storage Architecture - Draft Document Tutorial

The timing sequence is: Host (DataIn) → Send CMD17 command → SD card (DataOut) → Return R1 response → SD card sends data block + CRC → Transaction ends.

20 File System and Storage Architecture - Draft Document Tutorial

The timing sequence is as follows: Host (DataIn) → sends CMD18 command → SD card (DataOut) → returns R1 response → SD card continuously sends multiple data blocks + CRC → Host sends CMD12 (stop command) → SD card returns R1 response → Transaction ends.

Similarly, SD cards support Single Block Write and Multiple Block Write commands, which correspond respectively to CMD24 and CMD25 commands in the SD card protocol:

20 File System and Storage Architecture - Draft Document Tutorial

Host (DataIn) → sends CMD24 command → SD card (DataOut) → returns R1 response → Host sends start block token + data block → SD card returns data response + busy signal (indicating writing in progress) → Transaction ends.

20 File System and Storage Architecture - Draft Document Tutorial

Host (DataIn) → sends CMD25 command → SD card (DataOut) → returns R1 response → Host continuously sends start block token + multiple data blocks → Host sends stop token → SD card returns busy signal → Transaction ends.

In SPI mode, the formats of commands, responses and data of the SD card are different from those in the standard SD mode. Each command is 6 bytes long, starting from 01 (start bit), followed by the command number (6 bits), then a 32-bit parameter, and finally a 7-bit CRC check and a 1-bit stop bit:

20 File System and Storage Architecture - Draft Document Tutorial
Bit positionLength (bits)FieldDescription
471Start bitFixed to 0
461Transmission bitFixed to 1 (indicates host → SD card)
21:406Command index6-bit command number (e.g., CMD0 corresponds to 0x00)
15:0832ParameterConfiguration information carried with the command (e.g., block address)
7:017CRC7CRC7 checksum of the first 41 bits
01Stop bitFixed to 1

For specific command functions and parameters, please refer to Table 7-3: Commands and Arguments in the SD card specification:

20 File System and Storage Architecture - Draft Document Tutorial
Command indexSPI supportCore functionResponse type
CMD0YesReset the SD card and enter idle stateR1
CMD8YesSend voltage condition, check compatibilityR7
CMD17YesSingle-block readR1
CMD18YesMulti-block readR1
CMD24YesSingle-block writeR1
CMD25YesMulti-block writeR1
CMD55YesIndicates the following command is an ACMD commandR1
CMD58YesRead the OCR registerR3

Application Commands ACMD For the functions and parameters, please refer to Table 7-4 in the specification: Application Specific Commands used/reserved by SD Memory Card - SPI Mode:

20 File System and Storage Architecture - Draft Document Tutorial

Response RESPONSE is typically 1 byte, referred to as the R1 response; in some cases, a response of 2 bytes or more may be required.

The structure of the R1 response is shown in the following figure, which includes an idle state indication bit, an erase reset indication bit, an illegal command indication bit, etc. :

20 File System and Storage Architecture - Draft Document Tutorial

The R2 response consists of 2 bytes: the first byte is R1, and the second byte contains additional error flags and some other flags.

20 File System and Storage Architecture - Draft Document Tutorial

The R3 response is used for OCR register read, and contains the R1 response and OCR data:

20 File System and Storage Architecture - Draft Document Tutorial

There is also an R7 response, which corresponds to the SEND_IF_COND command (CMD8). This response is 5 bytes long: the first byte has the same structure as the R1-type response, used to indicate the command status and error information, while the remaining four bytes contain the card's operating voltage information and the echo of the check pattern:

20 File System and Storage Architecture - Draft Document Tutorial

The different responses are compared as follows:

Response typeR1 responseR2 responseR3 responseR7 response
Length1 byte2 bytes5 bytes5 bytes
Structure8 bits:
Bit 0 = parameter error
Bit 1 = address error
Bit 2 = erase sequence error
Bit 3 = command CRC error
Bit 4 = illegal command
Bit 5 = erase reset
Bit 6 = idle state
Bit 7 = fixed to 0
2 bytes:
Byte 1 = R1 response
Byte 2 = extended status (card locked, write protect violation, etc.)
5 bytes:
Byte 1 = R1 response
Bytes 2-5 = OCR register data (operating voltage, capacity identifier)
5 bytes:
Byte 1 = R1 response
Bytes 2-5 = voltage accepted flag + check pattern echo (corresponds to the CMD8 argument)
PurposeResponse to most CMD commands; indicates the current state of the SD cardResponse when returning CID/CSD register dataResponse to the CMD58 command; returns OCR register informationResponse to the CMD8 command; checks voltage compatibility

A data token is the unit that carries the actual data in SD card read and write operations, and its general structure follows the format of "start flag + data + checksum", while complying with fixed transmission rules:

Depending on the type of read/write operation (single-block or multi-block), the start flag of the data token uses different values to correspond to different transmission scenarios:

Operation scenarioStart flag
(1 byte)
Binary format
(corresponds to the figure)
Data segment lengthChecksumApplicable commands
Single-block read / write0xFE11111110
(Figure 1: bits 7-1 are 1, bit 0 is 0)
2~513 bytes (fixed at 512 bytes for SDHC/SDXC)2-byte CRC16CMD17 (single-block read), CMD24 (single-block write)
Start of multi-block read / write0xFC11111100
(Figure 2: Start Block Token)
2~513 bytes (fixed at 512 bytes for SDHC/SDXC)2-byte CRC16CMD18 (multi-block read), CMD25 (multi-block write)
Terminate multi-block transfer0xFD11111101 (
Figure 2: Stop Tran Token)
- (no data field)-Used with CMD12 to terminate multi-block transfer
20 File System and Storage Architecture - Draft Document Tutorial

When performing write and read operations on multiple data tokens, first use 0xFC to mark the start of a data token, and use 0xFD at the end of data transmission:

20 File System and Storage Architecture - Draft Document Tutorial

After the data transmission is completed, the SD card feeds back the result via a response token/error token:

Feedback typeToken format (1 byte)Function descriptionApplicable scenarios
Data response tokenBits 7-4: any value;
Bit 4 = 0;
Bits 3-1 = status bits;
Bit 0 = 1 (Figure 3)
Reports the result of the data writeAfter single-block write and multi-block write operations
Data error tokenBits 7-4 = 0000;
Bits 3-0 = error bits (Figure 4)
Reports the reason a read operation failed (same as the error bits in the R2 response)When a read operation fails

Here is the data response token:

20 File System and Storage Architecture - Draft Document Tutorial

The definition of the status bits is as follows:

In multi-block write operations, if any error occurs (CRC error or write error), the host shall stop the data transmission with CMD12; if a write error occurs (with a response of '110'), the host may send CMD13 (SEND_STATUS) to obtain the cause of the write issue, or identify the number of successfully written blocks via ACMD22.

If a read operation fails and the SD card cannot provide the required data, the card will send a 1-byte data error token, whose format is as follows:

20 File System and Storage Architecture - Draft Document Tutorial

Among them, the 4 least significant bits (LSB) are the same as the error bits in the R2 response format, indicating different error states.

3.4.1.5 Introduction to the Storage Structure of SD Card

Through an external communication interface (SD bus or SPI bus), we can access the memory cells of an SD card:

20 File System and Storage Architecture - Draft Document Tutorial

The storage structure of an SD card consists of two parts: the internal hardware architecture and the logical/physical storage units.

3.4.2 Implementation of Custom SD Card Read/Write Class

In the following code, we implement functions such as SD card initialization, command sending, and data reading, and interact with the SD card through the SPI communication interface to ensure that the card can work properly.

The sample code is as follows:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/30 PM 12:11
# @Author  : Li Qingshui
# @File    : sdcard.py       
# @Description : Custom SDCard class for SD card reading and writing
# Reference code: https://github.com/micropython/micropython-lib/blob/master/micropython/drivers/storage/sdcard/sdcard.py#L291

# ======================================== Import Modules ========================================

# Import MicroPython-related modules
from micropython import const
# Import time-related modules
import time
# Import hardware-related modules
from machine import SPI, Pin

# ======================================== Global Variables ============================================

# Define SD card related constants
CMD_TIMEOUT = const(100)            # Command timeout constant
R1_IDLE_STATE = const(1 << 0)       # R1 response status: idle state
R1_ILLEGAL_COMMAND = const(1 << 2)  # R1 response status: illegal command
TOKEN_CMD25 = const(0xFC)           # Data transfer command token
TOKEN_STOP_TRAN = const(0xFD)       # Stop transmission token
TOKEN_DATA = const(0xFE)            # Data token

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

# Custom SD card operation class
class SDCard:
    """
    SDCard class, used to operate SD/MMC memory cards over the SPI bus.
    This class encapsulates the low-level SD card communication protocol and supports initialization of standard capacity cards (SDSC) and high capacity cards (SDHC/SDXC),
    block read/write and erase operations. It implements the complete SPI-mode protocol stack, including card identification mode and data transfer mode.

    Attributes:
        spi (SPI): SPI interface instance, used to communicate with the SD card
        cs (Pin): Chip select pin instance, used to select the SD card
        sectors (int): Total number of sectors on the card (read-only)
        cdv (int): Block addressing factor (1 means block addressing, 512 means byte addressing)
        cmdbuf (bytearray): Command buffer (6 bytes)
        dummybuf (bytearray): Data buffer (512 bytes)
        tokenbuf (bytearray): Token buffer (1 byte)

    Methods:
        __init__(self, spi: machine.SPI, cs: machine.Pin, baudrate: int = 1320000) -> None:
            Initialize the SD card controller

        init_spi(self, baudrate: int) -> None:
            Configure SPI interface parameters

        init_card(self, baudrate: int) -> None:
            Execute the complete initialization procedure

        cmd(self, cmd: int, arg: int, crc: int, final: int = 0, release: bool = True, skip1: bool = False) -> int:
            Send an SD card command and get the response

        readinto(self, buf: bytearray) -> None:
            Read a data block into the buffer

        write(self, token: int, buf: bytearray) -> bool:
            Write a data block to the card

        write_token(self, token: int) -> None:
            Send a control token

        erase_block(self, block_number: int) -> None:
            Erase the specified data block

        init_card_v1(self) -> None:
            Initialize a standard capacity SD card

        init_card_v2(self) -> None:
            Initialize a high capacity SD card

    Constants:
        CMD_TIMEOUT: Number of command timeout retries (100)
        R1_IDLE_STATE: Card idle state flag (0x01)
        R1_ILLEGAL_COMMAND: Illegal command flag (0x04)
        TOKEN_CMD25: Multi-block write start token (0xFC)
        TOKEN_STOP_TRAN: Stop transmission token (0xFD)
        TOKEN_DATA: Single-block data token (0xFE)
    """
    def __init__(self, spi: SPI, cs: Pin, baudrate: int = 1320000) -> None:
        """
        Initialize the SD card controller.

        Args:
            spi (machine.SPI): Configured SPI interface object
            cs (machine.Pin): Chip select pin object
            baudrate (int, optional): SPI baud rate, default 1320000

        Returns:
            None

        Raises:
            OSError: If a hardware error occurs during initialization
            ValueError: If the baud rate is out of the valid range
        """
        # Store the SPI interface
        self.spi = spi
        # Store the CS pin
        self.cs = cs

        # Create the command buffer, 6 bytes long
        self.cmdbuf = bytearray(6)
        # Create the dummy data buffer, 512 bytes long
        self.dummybuf = bytearray(512)
        # Create the token buffer, 1 byte long
        self.tokenbuf = bytearray(1)

        # Initialize the dummy data buffer to all FF
        for i in range(512):
            self.dummybuf[i] = 0xFF

        # Create a memory view of the dummy data buffer
        self.dummybuf_memoryview = memoryview(self.dummybuf)

        # Initialize the SD card
        self.init_card(baudrate)

    def init_spi(self, baudrate: int) -> None:
        """
        Configure the SPI interface parameters.

        Args:
            baudrate (int): Target communication rate (Hz)

        Returns:
            None

        Raises:
            ValueError: If the baud rate is out of the range supported by the hardware
        """
        # Initialize SPI
        self.spi.init(baudrate=baudrate, phase=0, polarity=0)

    def init_card(self, baudrate: int) -> None:
        """
        Perform the complete SD card initialization sequence.

        Args:
            baudrate (int): Target data transfer baud rate

        Returns:
            None

        Raises:
            OSError: If any of the following occurs:
                - Card does not respond
                - Version identification fails
                - CSD format is not supported
                - Block size setting fails
        """
        # Initialize the CS pin, set the CS pin to output high
        self.cs.init(self.cs.OUT, value=1)

        # Initialize the SPI bus; initialize at a low data rate
        self.init_spi(100000)

        # Idle cycles before sending commands, keep CS high
        for i in range(16):
            # Send 16 null bytes
            self.spi.write(b"\xff")

        # CMD0: Initialize the card; should return _IDLE_STATE (5 attempts allowed)
        for _ in range(5):
            if self.cmd(0, 0, 0x95) == R1_IDLE_STATE:
                break
        else:
            # If no SD card is found, raise an exception
            raise OSError("no SD card")

        # CMD8: Determine the card version
        r = self.cmd(8, 0x01AA, 0x87, 4)
        if r == R1_IDLE_STATE:
            # Initialize a v2 card
            self.init_card_v2()
        elif r == (R1_IDLE_STATE | R1_ILLEGAL_COMMAND):
            # Initialize a v1 card
            self.init_card_v1()
        else:
            # Unable to determine the SD card version
            raise OSError("couldn't determine SD card version")

        # Get the number of sectors
        # CMD9: Response R2 (R1 byte + 16-byte block read)
        if self.cmd(9, 0, 0, 0, False) != 0:
            # If there is no response, raise an exception
            raise OSError("no response from SD card")

        # Create the CSD buffer
        csd = bytearray(16)
        # Read the CSD data
        self.readinto(csd)

        # CSD version 2.0
        if csd[0] & 0xC0 == 0x40:
            # Calculate the number of sectors
            self.sectors = ((csd[8] << 8 | csd[9]) + 1) * 1024
            # CSD version 1.0 (old, <=2GB)
        elif csd[0] & 0xC0 == 0x00:
            # Get C_SIZE
            c_size = (csd[6] & 0b11) << 10 | csd[7] << 2 | csd[8] >> 6
            # Get C_SIZE_MULT
            c_size_mult = (csd[9] & 0b11) << 1 | csd[10] >> 7
            # Get the read block length
            read_bl_len = csd[5] & 0b1111
            # Calculate the capacity
            capacity = (c_size + 1) * (2 ** (c_size_mult + 2)) * (2**read_bl_len)
            # Calculate the number of sectors
            self.sectors = capacity // 512
        else:
            # Unsupported CSD format
            raise OSError("SD card CSD format not supported")

        # CMD16: Set the block length to 512 bytes
        if self.cmd(16, 512, 0) != 0:
            # Unable to set the block size
            raise OSError("can't set 512 block size")

        # Now switch to a high data rate
        self.init_spi(baudrate)

    def init_card_v1(self) -> None:
        """
        Initialize a standard capacity SD card (SDSC).

        Args:
            None

        Returns:
            None

        Raises:
            OSError: If initialization times out (100 attempts)
        """
        for i in range(CMD_TIMEOUT):
            time.sleep_ms(50)
            # Send command 55
            self.cmd(55, 0, 0)

            # Try to initialize the card
            # SDSC card, use byte addressing for read/write/erase commands
            if self.cmd(41, 0, 0) == 0:
                # Set the block size to 512 bytes
                self.cdv = 512
                # Return on success
                return
        # Raise an exception on timeout
        raise OSError("timeout waiting for v1 card")

    def init_card_v2(self) -> None:
        """
        Initialize a high capacity SD card (SDHC/SDXC).

        Args:
            None

        Returns:
            None

        Raises:
            OSError: If initialization times out (100 attempts)
        """
        for i in range(CMD_TIMEOUT):
            # Wait 50 ms
            time.sleep_ms(50)
            # Send command 58 to get the OCR
            self.cmd(58, 0, 0, 4)
            # Send command 55 to prepare for initialization
            self.cmd(55, 0, 0)

            # Try to initialize the SDHC/SDXC card
            if self.cmd(41, 0x40000000, 0) == 0:
                # 4-byte response, a negative value means keep the first byte
                self.cmd(58, 0, 0, -4)
                # Get the first byte of the OCR
                ocr = self.tokenbuf[0]
                # SDSC card, use byte addressing
                if not ocr & 0x40:
                    # Set the block size to 512 bytes
                    self.cdv = 512
                else:
                    # SDHC/SDXC card, use block addressing
                    # Set the block size to 1 byte
                    self.cdv = 1
                # Return on success
                return

        # Raise an exception on timeout
        raise OSError("timeout waiting for v2 card")

    def cmd(self, cmd: int, arg: int, crc: int, final: int = 0, release: bool = True, skip1: bool = False) -> int:
        """
        Send an SD card command and get the response.

        Args:
            cmd (int): Command number (0-63)
            arg (int): 32-bit command argument
            crc (int): CRC check value
            final (int, optional): Number of extra bytes to read, default 0
            release (bool, optional): Whether to release CS, default True
            skip1 (bool, optional): Whether to skip the first byte, default False

        Returns:
            int: Response status:
                - 0x00: Normal
                - 0x01: Idle state
                - -1: Timeout

        Raises:
            OSError: If SPI communication fails
        """
        # Pull the CS pin low to start communication
        self.cs(0)

        # Create and send the command
        buf = self.cmdbuf

        # Set the command header
        buf[0] = 0x40 | cmd
        # Set the command argument (high byte)
        buf[1] = arg >> 24
        # Set the command argument (second-highest byte)
        buf[2] = arg >> 16
        # Set the command argument (second-lowest byte)
        buf[3] = arg >> 8
        # Set the command argument (low byte)
        buf[4] = arg
        # Set the CRC check
        buf[5] = crc

        # Send the command over SPI
        self.spi.write(buf)

        # If the first response needs to be skipped
        if skip1:
            # Read one byte and discard it
            self.spi.readinto(self.tokenbuf, 0xFF)

        # Wait for the response (response[7] == 0)
        for i in range(CMD_TIMEOUT):
            # Read the response byte
            self.spi.readinto(self.tokenbuf, 0xFF)
            # Get the response
            response = self.tokenbuf[0]

            # Check whether the response is valid
            if not (response & 0x80):
                # Handle the big-endian integer case
                # If final is negative
                if final < 0:
                    # Read and discard one byte
                    self.spi.readinto(self.tokenbuf, 0xFF)
                    # Update the final value
                    final = -1 - final

                # Read the extra bytes
                for j in range(final):
                    # Send a null byte to maintain timing
                    self.spi.write(b"\xff")

                # If the CS signal needs to be released
                if release:
                    # Pull the CS pin high
                    self.cs(1)
                    # Send a null byte
                    self.spi.write(b"\xff")
                # Return the valid response
                return response

        # Timeout handling
        # Pull the CS pin high
        self.cs(1)
        # Send a null byte
        self.spi.write(b"\xff")
        # Return -1 to indicate timeout
        return -1

    def readinto(self, buf: bytearray) -> None:
        """
        Read a data block into the buffer.

        Args:
            buf (bytearray): Target buffer (must be 512 bytes)

        Returns:
            None

        Raises:
            ValueError: If the buffer length is wrong
            OSError: If the read times out or the check fails
        """
        # Pull the CS pin low to start communication
        self.cs(0)

        # Read until the start byte (0xff)
        for i in range(CMD_TIMEOUT):
            # Read one byte
            self.spi.readinto(self.tokenbuf, 0xFF)
            # Check whether it is the data start token
            if self.tokenbuf[0] == TOKEN_DATA:
                # Found the start byte, exit the loop
                break
            # Wait 1 ms
            time.sleep_ms(1)
        else:
            # Pull the CS pin high
            self.cs(1)
            # Raise a timeout error
            raise OSError("timeout waiting for response")

        # Read data, create a memory view
        mv = self.dummybuf_memoryview
        # If the buffer length does not match
        if len(buf) != len(mv):
            # Slice to the buffer length
            mv = mv[: len(buf)]
        # Read data into buf
        self.spi.write_readinto(mv, buf)

        # Send a null byte
        self.spi.write(b"\xff")
        self.spi.write(b"\xff")

        # Pull the CS pin high
        self.cs(1)
        # Send a null byte
        self.spi.write(b"\xff")

    def write(self, token: int, buf: bytearray) -> bool:
        """
        Write a data block to the card.

        Args:
            token (int): Write token (0xFC/0xFD/0xFE)
            buf (bytearray): Source data (must be 512 bytes)

        Returns:
            bool: Write result:
                - True: Success
                - False: Failure

        Raises:
            ValueError: If the data length is wrong
            OSError: If the card responds abnormally
        """
        # Pull the CS pin low to start communication
        self.cs(0)

        # Send: block start marker, data, checksum
        self.spi.read(1, token)
        self.spi.write(buf)
        self.spi.write(b"\xff")
        self.spi.write(b"\xff")

        # Check whether the response is the success flag
        if (self.spi.read(1, 0xFF)[0] & 0x1F) != 0x05:
            # Pull the CS pin high
            self.cs(1)
            # Send a null byte
            self.spi.write(b"\xff")
            # Return, write failed
            return

        # Wait for the write to complete
        while self.spi.read(1, 0xFF)[0] == 0:
            # Wait until the write is complete
            pass

        # Pull the CS pin high
        self.cs(1)
        # Send a null byte
        self.spi.write(b"\xff")

    def write_token(self, token: int) -> None:
        """
        Send a control token.

        Args:
            token (int): Control token (0xFC/0xFD/0xFE)

        Returns:
            None

        Raises:
            OSError: If the card does not respond
        """
        # Pull the CS pin low to start communication
        self.cs(0)
        # Send the marker
        self.spi.read(1, token)
        # Send a null byte
        self.spi.write(b"\xff")

        # Wait for the write to complete
        while self.spi.read(1, 0xFF)[0] == 0x00:
            # Wait until the write is complete
            pass

        # Pull the CS pin high
        self.cs(1)
        # Send a null byte
        self.spi.write(b"\xff")

    def erase_block(self, block_number: int) -> None:
        """
        Erase the specified data blocks.

        Args:
            block_number (int): Block number (LBA format)

        Returns:
            None

        Raises:
            ValueError: If the block number is out of range
            OSError: If the erase command fails
        """
        # Pull the CS pin low to start communication
        self.cs(0)

        # Send erase commands CMD32 (erase start) and CMD33 (erase end)
        # Start erase
        self.cmd(32, block_number * 512, 0)
        # End erase
        self.cmd(33, block_number * 512, 0)

        # Send erase command CMD38 (erase)
        self.cmd(38, 0, 0)

        # Wait for the erase to complete
        while True:
            # Query status
            response = self.cmd(13, 0, 0)
            # Erase complete
            if response == 0:
                break

        # Pull the CS pin high
        self.cs(1)
        # Send a null byte
        self.spi.write(b"\xff")

# ======================================== Initialization ==========================================

# ========================================  Main Program  ============================================

memoryview memory view

memoryview is a memory view object provided by Python, which allows accessing and manipulating mutable buffer objects (such as bytearray, bytes, array.array) without copying data. Its main function is to improve data processing efficiency and reduce unnecessary memory usage.

In Python, when working with binary data (such as bytearray):

  • Direct slicing creates a copy of the data, which consumes additional memory.
  • memoryview allows direct manipulation of the original data, avoiding copying and improving performance.

In the classes mentioned above, if you directly use bytearray, each slicing operation (such as dummybuf[: len (buf)]) will create a new copy, which impacts performance; in contrast, memoryview directly references the original data without making copies.

For more information about memoryview, please refer to the following document: https://docs.python.org/3.5/library/stdtypes.html#memoryview

3.4.2.1 Initialization Method

In the custom SDCard class, initialization is performed first: configure SPI communication and the CS pin, and create multiple buffers to support subsequent operations:

def __init__(self, spi: SPI, cs: Pin, baudrate: int = 1320000) -> None:
    """
    Initialize the SD card controller.

    Args:
        spi (machine.SPI): Configured SPI interface object
        cs (machine.Pin): Chip select pin object
        baudrate (int, optional): SPI baud rate, default 1320000

    Returns:
        None

    Raises:
        OSError: If a hardware error occurs during initialization
        ValueError: If the baud rate is out of the valid range
    """
    # Store the SPI interface
    self.spi = spi
    # Store the CS pin
    self.cs = cs

    # Create the command buffer, 6 bytes long
    self.cmdbuf = bytearray(6)
    # Create the dummy data buffer, 512 bytes long
    self.dummybuf = bytearray(512)
    # Create the token buffer, 1 byte long
    self.tokenbuf = bytearray(1)

    # Initialize the dummy data buffer to all FF
    for i in range(512):
        self.dummybuf[i] = 0xFF

    # Create a memory view of the dummy data buffer
    self.dummybuf_memoryview = memoryview(self.dummybuf)

    # Initialize the SD card
    self.init_card(baudrate)

The specific workflow is as follows:

20 File System and Storage Architecture - Draft Document Tutorial

The actual method for initializing the SD card is the init_card method, and this function is responsible for detecting and initializing the SD card as well as determining the card type (SDSC or SDHC/SDXC):

20 File System and Storage Architecture - Draft Document Tutorial

First, set the chip select pin CS to high level (1), indicating that the SD card is not selected:

def init_card(self, baudrate: int) -> None:
    """
    Perform the complete SD card initialization sequence.

    Args:
        baudrate (int): Target data transfer baud rate

    Returns:
        None

    Raises:
        OSError: If any of the following occurs:
            - Card does not respond
            - Version identification fails
            - CSD format is not supported
            - Block size setting fails
    """
    # Initialize the CS pin, set the CS pin to output high
    self.cs.init(self.cs.OUT, value=1)

Then initialize the SPI at a low speed (100kHz), since the SD card requires low-speed communication after power-on:

# Initialize the SPI bus; initialize at a low data rate
self.init_spi(100000)

Then send 16 bytes 0xFF to allow the SD card to complete its internal power-on and enter the SPI communication mode:

# Idle cycles before sending commands, keep CS high
for i in range(16):
    # Send 16 null bytes
    self.spi.write(b"\xff")

Then send CMD0 in a loop for 5 times to enter SPI mode, and throw OSError ("no SD card") if the expected response is not obtained.

Then send CMD8 to check the SD version:

# CMD8: Determine the card version
r = self.cmd(8, 0x01AA, 0x87, 4)
if r == R1_IDLE_STATE:
    # Initialize a v2 card
    self.init_card_v2()
elif r == (R1_IDLE_STATE | R1_ILLEGAL_COMMAND):
    # Initialize a v1 card
    self.init_card_v1()
else:
    # Unable to determine the SD card version
    raise OSError("couldn't determine SD card version")

New SD cards (≥2GB) support either block addressing mode or byte addressing mode, and the init_card_v2 () function performs the following operations:

20 File System and Storage Architecture - Draft Document Tutorial
def init_card_v2(self) -> None:
    """
    Initialize a high capacity SD card (SDHC/SDXC).

    Args:
        None

    Returns:
        None

    Raises:
        OSError: If initialization times out (100 attempts)
    """
    for i in range(CMD_TIMEOUT):
        # Wait 50 ms
        time.sleep_ms(50)
        # Send command 58 to get the OCR
        self.cmd(58, 0, 0, 4)
        # Send command 55 to prepare for initialization
        self.cmd(55, 0, 0)

        # Try to initialize the SDHC/SDXC card
        if self.cmd(41, 0x40000000, 0) == 0:
            # 4-byte response, a negative value means keep the first byte
            self.cmd(58, 0, 0, -4)
            # Get the first byte of the OCR
            ocr = self.tokenbuf[0]
            # SDSC card, use byte addressing
            if not ocr & 0x40:
                # Set the block size to 512 bytes
                self.cdv = 512
            else:
                # SDHC/SDXC card, use block addressing
                # Set the block size to 1 byte
                self.cdv = 1
            # Return on success
            return

    # Raise an exception on timeout
    raise OSError("timeout waiting for v2 card")

Legacy SD cards (≤2GB) all use byte addressing mode, send CMD55 + CMD41 in a loop to wait for the SD card to complete initialization, then set the block addressing factor cdv = 1:

20 File System and Storage Architecture - Draft Document Tutorial
def init_card_v1(self) -> None:
    """
    Initialize a standard capacity SD card (SDSC).

    Args:
        None

    Returns:
        None

    Raises:
        OSError: If initialization times out (100 attempts)
    """
    for i in range(CMD_TIMEOUT):
        time.sleep_ms(50)
        # Send command 55
        self.cmd(55, 0, 0)

        # Try to initialize the card
        # SDSC card, use byte addressing for read/write/erase commands
        if self.cmd(41, 0, 0) == 0:
            # Set the block size to 512 bytes
            self.cdv = 512
            # Return on success
            return
    # Raise an exception on timeout
    raise OSError("timeout waiting for v1 card")

Then the init_card method sends CMD9 (to read the 16-byte CSD register data) to obtain capacity information:

# Get the number of sectors
# CMD9: Response R2 (R1 byte + 16-byte block read)
if self.cmd(9, 0, 0, 0, False) != 0:
    # If there is no response, raise an exception
    raise OSError("no response from SD card")

# Create the CSD buffer
csd = bytearray(16)
# Read the CSD data
self.readinto(csd)

Then calculate the number of sectors of the SD card:

# CSD version 2.0
if csd[0] & 0xC0 == 0x40:
    # Calculate the number of sectors
    self.sectors = ((csd[8] << 8 | csd[9]) + 1) * 1024
    # CSD version 1.0 (old, <=2GB)
elif csd[0] & 0xC0 == 0x00:
    # Get C_SIZE
    c_size = (csd[6] & 0b11) << 10 | csd[7] << 2 | csd[8] >> 6
    # Get C_SIZE_MULT
    c_size_mult = (csd[9] & 0b11) << 1 | csd[10] >> 7
    # Get the read block length
    read_bl_len = csd[5] & 0b1111
    # Calculate the capacity
    capacity = (c_size + 1) * (2 ** (c_size_mult + 2)) * (2**read_bl_len)
    # Calculate the number of sectors
    self.sectors = capacity // 512
else:
    # Unsupported CSD format
    raise OSError("SD card CSD format not supported")

After setting the block length to 512 bytes, the initialization is completed, and the SPI settings are adjusted to achieve a high data transfer rate:

# CMD16: Set the block length to 512 bytes
if self.cmd(16, 512, 0) != 0:
    # Unable to set the block size
    raise OSError("can't set 512 block size")

# Now switch to a high data rate
self.init_spi(baudrate)
3.4.2.2 Command Sending Method

The cmd method is used to send commands to the SD card and obtain responses, and the entire process can be divided into five main stages:

20 File System and Storage Architecture - Draft Document Tutorial
  1. Pull down the CS (Chip Select) pin to initiate SPI communication;
  2. Construct and send the command;
  3. Wait for the SD card to respond (return -1 on timeout);
  4. Read optional additional data;
  5. Release CS and return the response.

Here, we first pull the CS pin low to start communication, then construct a command buffer containing the command number, parameters and CRC check value, and send the command via SPI:

def cmd(self, cmd: int, arg: int, crc: int, final: int = 0, release: bool = True, skip1: bool = False) -> int:
    """
    Send an SD card command and get the response.

    Args:
        cmd (int): Command number (0-63)
        arg (int): 32-bit command argument
        crc (int): CRC check value
        final (int, optional): Number of extra bytes to read, default 0
        release (bool, optional): Whether to release CS, default True
        skip1 (bool, optional): Whether to skip the first byte, default False

    Returns:
        int: Response status:
            - 0x00: Normal
            - 0x01: Idle state
            - -1: Timeout

    Raises:
        OSError: If SPI communication fails
    """
    # Pull the CS pin low to start communication
    self.cs(0)

    # Create and send the command
    buf = self.cmdbuf

    # Set the command header
    buf[0] = 0x40 | cmd
    # Set the command argument (high byte)
    buf[1] = arg >> 24
    # Set the command argument (second-highest byte)
    buf[2] = arg >> 16
    # Set the command argument (second-lowest byte)
    buf[3] = arg >> 8
    # Set the command argument (low byte)
    buf[4] = arg
    # Set the CRC check
    buf[5] = crc

    # Send the command over SPI
    self.spi.write(buf)

If the option to skip the first response is enabled, the first byte will be discarded. Then, the system enters a loop to read the response from the SD card until a valid response is received or a timeout occurs. If the response is valid, any additional bytes will be read as required and the CS signal will be deasserted:

# If the first response needs to be skipped
if skip1:
    # Read one byte and discard it
    self.spi.readinto(self.tokenbuf, 0xFF)

# Wait for the response (response[7] == 0)
for i in range(CMD_TIMEOUT):
    # Read the response byte
    self.spi.readinto(self.tokenbuf, 0xFF)
    # Get the response
    response = self.tokenbuf[0]

    # Check whether the response is valid
    if not (response & 0x80):
        # Handle the big-endian integer case
        # If final is negative
        if final < 0:
            # Read and discard one byte
            self.spi.readinto(self.tokenbuf, 0xFF)
            # Update the final value
            final = -1 - final

        # Read the extra bytes
        for j in range(final):
            # Send a null byte to maintain timing
            self.spi.write(b"\xff")

        # If the CS signal needs to be released
        if release:
            # Pull the CS pin high
            self.cs(1)
            # Send a null byte
            self.spi.write(b"\xff")
        # Return the valid response
        return response

Finally, the response bytes of the SD card are returned, or -1 is returned in case of a timeout:

# Timeout handling
# Pull the CS pin high
self.cs(1)
# Send a null byte
self.spi.write(b"\xff")
# Return -1 to indicate timeout
return -1
3.4.2.3 Data Block Reading Method

The readinto method is used to read a 512-byte data block from the SD card and store it into the specified buffer. First, pull the CS pin low to initiate communication, then enter a loop to continuously read bytes until the data start marker (0xFF) is detected. If the start byte is not found within the set timeout period, pull the CS pin high and throw a timeout error:

def readinto(self, buf: bytearray) -> None:
    """
    Read a data block into the buffer.

    Args:
        buf (bytearray): Target buffer (must be 512 bytes)

    Returns:
        None

    Raises:
        ValueError: If the buffer length is wrong
        OSError: If the read times out or the check fails
    """
    # Pull the CS pin low to start communication
    self.cs(0)

    # Read until the start byte (0xff)
    for i in range(CMD_TIMEOUT):
        # Read one byte
        self.spi.readinto(self.tokenbuf, 0xFF)
        # Check whether it is the data start token
        if self.tokenbuf[0] == TOKEN_DATA:
            # Found the start byte, exit the loop
            break
        # Wait 1 ms
        time.sleep_ms(1)
    else:
        # Pull the CS pin high
        self.cs(1)
        # Raise a timeout error
        raise OSError("timeout waiting for response")

After finding the start byte, create a memory view and check whether the length of the buffer matches; if not, perform slicing. Then, read data from the SD card into the provided buffer via SPI and send two null bytes:

# Read data, create a memory view
mv = self.dummybuf_memoryview
# If the buffer length does not match
if len(buf) != len(mv):
    # Slice to the buffer length
    mv = mv[: len(buf)]
# Read data into buf
self.spi.write_readinto(mv, buf)

# Send a null byte
self.spi.write(b"\xff")
self.spi.write(b"\xff")

# Pull the CS pin high
self.cs(1)
# Send a null byte
self.spi.write(b"\xff")

The overall Sequence Diagram is shown as follows:

20 File System and Storage Architecture - Draft Document Tutorial
3.4.2.4 Data Block Reading Method

The write method is used to write a 512-byte data block to the SD card. First, pull the CS pin low to start communication, then send the write start token, data, and two null bytes:

def write(self, token: int, buf: bytearray) -> bool:
    """
    Write a data block to the card.

    Args:
        token (int): Write token (0xFC/0xFD/0xFE)
        buf (bytearray): Source data (must be 512 bytes)

    Returns:
        bool: Write result:
            - True: Success
            - False: Failure

    Raises:
        ValueError: If the data length is wrong
        OSError: If the card responds abnormally
    """
    # Pull the CS pin low to start communication
    self.cs(0)

    # Send: block start marker, data, checksum
    self.spi.read(1, token)
    self.spi.write(buf)
    self.spi.write(b"\xff")
    self.spi.write(b"\xff")

Next, check the response to confirm whether the write operation is successful. If the response does not match the success flag, pull the CS pin high, send a null byte and then return; if the response indicates success, the method will wait until the write operation is completed:

# Check whether the response is the success flag
if (self.spi.read(1, 0xFF)[0] & 0x1F) != 0x05:
    # Pull the CS pin high
    self.cs(1)
    # Send a null byte
    self.spi.write(b"\xff")
    # Return, write failed
    return

# Wait for the write to complete
while self.spi.read(1, 0xFF)[0] == 0:
    # Wait until the write is complete
    pass

After completion, pull the CS pin high again and send a null byte to terminate the communication:

# Pull the CS pin high
self.cs(1)
# Send a null byte
self.spi.write(b"\xff")

The overall Sequence Diagram is shown as follows:

20 File System and Storage Architecture - Draft Document Tutorial
3.4.2.5 Data Token Writing Method

write_token method is used to send a control token and wait for the SD card to complete internal operations. First, pull the CS pin low, send the start token and a null byte, then wait for the write operation to finish; finally, pull the CS pin high and send a null byte to terminate the communication:

def write_token(self, token: int) -> None:
    """
    Send a control token.

    Args:
        token (int): Control token (0xFC/0xFD/0xFE)

    Returns:
        None

    Raises:
        OSError: If the card does not respond
    """
    # Pull the CS pin low to start communication
    self.cs(0)
    # Send the marker
    self.spi.read(1, token)
    # Send a null byte
    self.spi.write(b"\xff")

    # Wait for the write to complete
    while self.spi.read(1, 0xFF)[0] == 0x00:
        # Wait until the write is complete
        pass

    # Pull the CS pin high
    self.cs(1)
    # Send a null byte
    self.spi.write(b"\xff")
3.4.2.6 Data Erasure Method

erase_block method is used to erase the specified data block on the SD card. First, pull the CS pin low to initiate communication, then send the erase commands CMD32 (erase start) and CMD33 (erase end) to specify the block number to be erased, and convert the block number to a byte offset by multiplying it by 512:

def erase_block(self, block_number: int) -> None:
    """
    Erase the specified data blocks.

    Args:
        block_number (int): Block number (LBA format)

    Returns:
        None

    Raises:
        ValueError: If the block number is out of range
        OSError: If the erase command fails
    """
    # Pull the CS pin low to start communication
    self.cs(0)

    # Send erase commands CMD32 (erase start) and CMD33 (erase end)
    # Start erase
    self.cmd(32, block_number * 512, 0)
    # End erase
    self.cmd(33, block_number * 512, 0)

Next, send CMD38 (erase command) to actually perform the erase operation, then enter a loop to continuously query the erase status by sending the CMD13 command until the return value is 0, which indicates that the erase is completed; finally, pull up the CS pin and send a dummy byte to terminate the communication:

# Wait for the erase to complete
while True:
    # Query status
    response = self.cmd(13, 0, 0)
    # Erase complete
    if response == 0:
        break

# Pull the CS pin high
self.cs(1)
# Send a null byte
self.spi.write(b"\xff")

3.4.3 SD Card Mounting File System

In the following code, we implement the operations of initializing the SD card and mounting it as a file system, and create a CSV file to write the website name and URL data. A CSV file can be thought of as a simple spreadsheet, but it does not have the rich functions and complex formats of Excel; instead, it is just a text file separated by commas. Each line in a CSV represents a record, the fields in a record are separated by commas, and its data is stored in plain text:

20 File System and Storage Architecture - Draft Document Tutorial
When opened with EXCEL, a CSV file is very similar to a spreadsheet file.
3.4.3.1 Preparations before the experiment

First, we need to insert the Fengya No. 1 Board - Ethernet and SD Card Expansion Board into the Fengya No. 1 Board - Universal Compatible Expansion Board, and set all the SW1 DIP switches on the Fengya No. 1 Board - Ethernet and SD Card Expansion Board to the ON position.

20 File System and Storage Architecture - Draft Document Tutorial

In addition, we also need to insert the SD card into the SD card slot on the SD card module. The SD card slot is equipped with internal electrical connections and control circuits, which can connect the SD card to the microcontroller.

20 File System and Storage Architecture - Draft Document Tutorial

The functions of each pin are as follows:

20 File System and Storage Architecture - Draft Document Tutorial

The function of each pin and its role in SPI mode are as follows:

Pin Number

Pin Name

Roles in SPI Mode

Pin1

DAT3/CS

chip select signal

Pin2

CMD/MOSI

Master output, slave input

Pin5

CLK/SCLK

SPI clock signal

Pin7

DAT0/MISO

Master input, slave output

Pin4

VDD/+3V3

power supply

Pin3

VSS/GND

grounding

For the rest, the other VSS pin Pin 6 needs to be grounded, while Pin 8 and Pin 9 need to be connected to a high level. It should be noted that the SPI bus MOSI, MISO, SCK and CS signals require pull-up resistors to ensure they remain at a high level when not being driven. Additionally, for the VDD power supply section, a 0.1µF ceramic capacitor and a 10µF electrolytic capacitor should be connected to filter the power supply.

The circuit schematic diagram is shown as follows:

20 File System and Storage Architecture - Draft Document Tutorial

The physical picture is shown as follows:

20 File System and Storage Architecture - Draft Document Tutorial

The connection between the SD card slot and the Raspberry Pi Pico is shown in the following table:

20 File System and Storage Architecture - Draft Document Tutorial
3.4.3.2 Implementation of SD Card Block Device Class

To implement file system mounting on an SD card, we first need to implement the block device required by the VFS (Virtual File System). Here, we have defined a SDCARDBlockDevice class for block device operations with the SD card, which provides functions for reading, writing, controlling and managing data blocks of the SD card. The sample code is as follows:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/10/1 AM 11:13
# @Author  : Li Qingshui
# @File    : sd_block_dev.py       
# @Description : Custom SD card block device class
# Reference code: https://github.com/micropython/micropython-lib/blob/master/micropython/drivers/storage/sdcard/sdcard.py#L291

# ======================================== Import Modules ========================================

# Import the abstract base class of the virtual file block device
from AbstractBlockDevInterface import AbstractBlockDev
# Import some constants defined in the custom SD card read/write class
from sdcard import TOKEN_CMD25,TOKEN_STOP_TRAN,TOKEN_DATA
# Import the custom SD card read/write class
from sdcard import SDCard

# ======================================== Global Variables ============================================

# Define error code constants, conforming to POSIX standard error codes
ENOERR = 0       # Operation successful
EPERM = 1        # Operation not permitted
EIO = 5          # I/O error
ENODEV = 19      # Invalid device/block number
EROFS = 30       # Read-only file system
EINVAL = 22      # Invalid argument
ENOSPC = 28      # No space

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

class SDCARDBlockDevice(AbstractBlockDev):
    """
    SDCARDBlockDevice class, implements read and write operations for an SD card block device.

    This class inherits from AbstractBlockDev and wraps block-level read and write operations on an SDCard device,
    providing the standard readblocks, writeblocks and ioctl methods, conforming to the MicroPython block device interface specification.

    Attributes:
        sdcard (SDCard): The associated SDCard instance, used to perform the actual SD card operations.

    Methods:
        __init__(self, sdcard: SDCard):
            Initialize an SDCARDBlockDevice instance and bind it to an SDCard device.

        readblocks(self, block_num: int, buf: bytearray, offset: int = 0):
            Read the data of one or more blocks from the SD card into a buffer.

        writeblocks(self, block_num: int, buf: bytearray, offset: int = 0):
            Write the buffer data to one or more blocks of the SD card.

        ioctl(self, op: int, arg: int) -> int:
            Device control interface, used to perform operations such as initialization, synchronization and getting block information.
    """
    def __init__(self, sdcard: SDCard) -> None:
        """
        Initialize an SDCARDBlockDevice instance.

        Args:
            sdcard (SDCard): The SDCard instance passed in.

        Raises:
            ValueError: If the sdcard passed in is not an SDCard instance.
        """

        # Check whether the SDCard instance passed in is valid
        if not isinstance(sdcard, SDCard):
            raise ValueError("Invalid SDCard instance")

        # Call the parent class constructor
        super().__init__()

        # Save the SDCard instance passed in
        self.sdcard = sdcard

    def readblocks(self, block_num: int, buf: bytearray, offset: int = 0) -> None:
        """
        Read data blocks from the SD card into a buffer.

        Args:
            block_num (int): Starting block number.
            buf (bytearray): Buffer used to store the read data; its length must be a multiple of 512 bytes.
            offset (int, optional): Data offset, default is 0.

        Raises:
            OSError: If the buffer length is not a multiple of 512, or the read fails.
        """
        # Check whether the buffer is valid and its length is 512 bytes
        if not buf or len(buf) % 512 != 0:
            raise OSError(EINVAL, "Buffer length must be multiple of 512")

        # Work around the shared bus issue: make sure MOSI is high before starting the transaction
        self.sdcard.spi.write(b"\xff")

        # Calculate the number of blocks to read
        nblocks = len(buf) // 512
        # Make sure the buffer length is valid
        assert nblocks and not len(buf) % 512, "Buffer length is invalid"

        if nblocks == 1:
            # CMD17: Set the read address of a single block
            if self.sdcard.cmd(17, block_num * self.sdcard.cdv, 0, release=False) != 0:
                # Release the card
                self.sdcard.cs(1)
                # EIO error
                raise OSError(EIO, "Failed to read singe block")
            # Receive data and release the card
            self.sdcard.readinto(buf)
        else:
            # CMD18: Set the read address of multiple blocks
            if self.sdcard.cmd(18, block_num * self.sdcard.cdv, 0, release=False) != 0:
                # Release the card
                self.sdcard.cs(1)
                # EIO error
                raise OSError(EIO, "Failed to read multi blocks")
            # Data offset
            offset = 0

            # Create a memory view
            mv = memoryview(buf)
            while nblocks:
                # Receive data and release the card
                self.sdcard.readinto(mv[offset : offset + 512])
                # Update the offset
                offset += 512
                # Decrease the remaining block count
                nblocks -= 1
            if self.sdcard.cmd(12, 0, 0xFF, skip1=True):
                # EIO error
                raise OSError(EIO, "Failed to stop multi blocks")

    def writeblocks(self, block_num: int, buf: bytearray, offset: int = 0) -> None:
        """
        Write data blocks from a buffer to the SD card.

        Args:
            block_num (int): Starting block number.
            buf (bytearray): Data to write; its length must be a multiple of 512 bytes.
            offset (int, optional): Data offset, default is 0.

        Raises:
            OSError: If the buffer length is not a multiple of 512, or the write fails.
        """
        # Check whether the buffer is valid and its length is 512 bytes
        if not buf or len(buf) % 512 != 0:
            raise OSError(EINVAL, "Buffer length must be multiple of 512")

        # Work around the shared bus issue: make sure MOSI is high before starting the transaction
        self.sdcard.spi.write(b"\xff")

        # Calculate the number of blocks to write
        nblocks, err = divmod(len(buf), 512)
        # Make sure the buffer length is valid
        assert nblocks and not err, "Buffer length is invalid"
        if nblocks == 1:
            # CMD24: Set the write address of a single block
            if self.sdcard.cmd(24, block_num * self.sdcard.cdv, 0) != 0:
                # EIO error
                raise OSError(EIO, "Failed to write singe block")

            # Send the data
            self.sdcard.write(TOKEN_DATA, buf)
        else:
            # CMD25: Set the write address of the first block
            if self.sdcard.cmd(25, block_num * self.sdcard.cdv, 0) != 0:
                # EIO error
                raise OSError(EIO, "Failed to write multi blocks")

            # Send the data
            offset = 0

            # Create a memory view
            mv = memoryview(buf)
            while nblocks:
                # Send each block of data
                self.sdcard.write(TOKEN_CMD25, mv[offset : offset + 512])
                # Update the offset
                offset += 512
                # Decrease the remaining block count
                nblocks -= 1
            # Send the stop transmission command
            self.sdcard.write_token(TOKEN_STOP_TRAN)

    def ioctl(self, op: int, arg: int) -> int:
        """
        Control the block device and query its parameters.

        Args:
            op (int): Operation code, see the AbstractBlockDev definition.
            arg (int): Additional argument.

        Returns:
            int: Operation result; returns 0 on success, or the corresponding block count/byte count.

        Raises:
            OSError: If the operation code is invalid or the erase block number is invalid.
        """
        # Initialize the device
        if op == AbstractBlockDev.IOCTL_INIT:
            # Perform the initialization operation
            self.sdcard.init_card(1320000)
            # Return 0 on success
            return ENOERR
        # Shut down the device
        elif op == AbstractBlockDev.IOCTL_SHUTDOWN:
            return ENOERR
        # Synchronize the device
        elif op == AbstractBlockDev.IOCTL_SYNC:
            return ENOERR
        # Get the number of blocks
        elif op == AbstractBlockDev.IOCTL_BLK_COUNT:
            # Return the number of sectors
            return self.sdcard.sectors
        # Get the block size (in bytes)
        elif op == AbstractBlockDev.IOCTL_BLK_SIZE:
            # Return 512 bytes
            return 512
        # Erase a block
        elif op == AbstractBlockDev.IOCTL_BLK_ERASE:
            # Erase the specified block
            if arg < 0 or arg >= self.sdcard.sectors:
                # If the block number is invalid, raise an error
                raise OSError(ENOSPC, "Invalid block number")
            self.sdcard.erase_block(arg)
            return ENOERR
        else:
            # Invalid operation code, raise an exception
            raise OSError(EINVAL, "Invalid ioctl operation")

# ======================================== Initialization ==========================================

# ========================================  Main Program  ============================================

This class provides the following methods:

20 File System and Storage Architecture - Draft Document Tutorial
20 File System and Storage Architecture - Draft Document Tutorial
20 File System and Storage Architecture - Draft Document Tutorial

3.4.3.3 Application Experiment

The following code can be found in the elegance-devkit v1\Demo\73 FileSys_SDCard folder in the resource package we provided.

In the main program, the code is as follows:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/30 PM 12:13
# @Author  : Li Qingshui
# @File    : main.py       
# @Description : File system class experiment: mount a file system on the SD card and read and write

# ======================================== Import Modules ========================================

# Import hardware-related modules
from machine import SPI, Pin
# Import time-related modules
import time
# Import the custom SD card block device class
from sd_block_dev import SDCARDBlockDevice
# Import the custom SD card read/write class
from sdcard import SDCard
# Import the virtual file class
import vfs
# Import the file system operation class
import os

# ======================================== Global Variables ============================================

# Define embedded knowledge learning websites and their URLs
websites = [
    ("Embedded.com", "https://www.embedded.com"),
    ("Microchip", "https://www.microchip.com"),
    ("ARM Developer", "https://developer.arm.com"),
    ("SparkFun", "https://www.sparkfun.com"),
    ("Adafruit", "https://www.adafruit.com"),
    ("Embedded Systems Academy", "https://www.esacademy.com"),
    ("Electronics Hub", "https://www.electronicshub.org"),
]

# Define the CSV file path
csv_file_path = '/sdcard/embedded_websites.csv'

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

# ======================================== Initialization ==========================================

# Power-on delay of 3 s
time.sleep(3)
# Print debug message
print("FreakStudio: Mount the SD Card to the file system")

# Initialize the SPI class, set the baud rate, polarity, phase, clock pin, and data pin
spi = SPI(1, baudrate=1320000, polarity=0, phase=0, sck=Pin(10), mosi=Pin(11), miso=Pin(12))
# Initialize the SD card class, using GPIO9 as the chip select pin
sdcard = SDCard(spi, cs=Pin(13))

# Create a block device using the SD card, with a block size of 512 bytes
block_device = SDCARDBlockDevice(sdcard = sdcard)
# Create a FAT file system on the block device
vfs.VfsFat.mkfs(block_device)
# Mount the block device to the /sdcard directory of the virtual file system
vfs.mount(block_device, '/sdcard')
# Print the current directory
print("Current Directory : ",os.listdir())

# ========================================  Main Program  ============================================

# Write the CSV file
with open(csv_file_path, 'w') as f:
    # Write the header
    f.write("Website Name,URL\n")
    for name, url in websites:
        # Write each row
        f.write(f"{name},{url}\n")

# Print the file location
print(f"CSV file written to SD card as '{csv_file_path}'.")

In the main program, we perform the following operations in sequence:

20 File System and Storage Architecture - Draft Document Tutorial
  1. Create an SPI instance, and configure the baud rate, polarity, phase and each pin (SCK, MOSI, MISO); create an SDCard instance, and use the specified SPI interface and GPIO as the chip select pin.
  2. Use SDCARDBlockDevice to encapsulate the SD card as a block device with a block size of 512 bytes.
  3. Use vfs.VfsFat.mkfs (block_device) to format the block device and create a FAT file system.
  4. Use vfs.mount (block_device, '/sdcard') to mount the block device to the /sdcard directory of the virtual file system.
  5. Use with open (csv_file_path, 'w') as f: to open the CSV file at the specified path in write mode, first write the header "Website Name, URL\n", then iterate through the website information using a loop, and write the name and URL of each website line by line.
  6. Finally, print a confirmation message indicating that the CSV file has been successfully written to the SD card, and provide the file path.

Burn the code, open the terminal, and the output is as follows:

20 File System and Storage Architecture - Draft Document Tutorial

As you can see, the CSV file has been successfully written to the /sdcard directory. To view its content, we need to remove the SD card and connect it to a computer via a card reader. Note that you should unmount the file system first at this point:

os.umount('/sdcard')

The unmount operation will forcibly flush all cached data to the SD card to ensure consistency between the physical storage and the actual data.

Meanwhile, note that you need to turn on the write-protect switch on the SD card to prevent the file system structure of the SD card from being altered after it is inserted into a computer, such as updates to the File Allocation Table (FAT), creation of hidden files (e. g. system files), or changes to file attributes.

After inserting the card reader into the computer, you will find a USB flash drive appears, which contains the CSV file we just wrote via the Raspberry Pi Pico:

20 File System and Storage Architecture - Draft Document Tutorial

Open the CSV file, and you can see that the Raspberry Pi Pico has successfully written the embedded learning website and its corresponding URL:

20 File System and Storage Architecture - Draft Document Tutorial

When you remove the SD card from the card reader and reinsert it into the SD card slot on the SD card module, you need to remount the file system:

vfs.mount(block_device, '/sdcard')

We can enter the following code in the REPL to verify whether the remount is successful and the files are kept intact:

# Read the CSV file and print its content
with open(csv_file_path, 'r') as f:
    for line in f:
        # Remove the trailing newline and split the fields
        parts = line.strip().split(',')
        if len(parts) == 2:
            print(f"Website: {parts[0]:<30} URL: {parts[1]}")
        else:
            # Print the header or other rows
            print(line.strip())

The terminal output is as follows, showing that the previously saved CSV file has been successfully read:

20 File System and Storage Architecture - Draft Document Tutorial

3.4 Application Experiment.mp4 16.0 MB

3.5 External EEPROM Mounted File System

3.5.1 Preparations before the experiment

We need to insert Fengya No. 1 Board - Environment and Storage Acquisition Board into Fengya No. 1 Board - Universal Compatible Expansion Board, and meanwhile turn on the SCL and SDA options of the SWITCH1 DIP switch on Fengya No. 1 Board - Environment and Storage Acquisition Board:

20 File System and Storage Architecture - Draft Document Tutorial

You can set its address by adjusting the Fengya No. 1 Board - Environment and Storage Acquisition Board via the ADDR DIP switches:

20 File System and Storage Architecture - Draft Document Tutorial

By default, the A2, A1 and A0 pins of the external EEPROM chip AT24C256 we use are all connected to GND (internal pull-down):

20 File System and Storage Architecture - Draft Document Tutorial

The device address is calculated as follows:

Device address=0b1010 0000+(0×22+0×21+0×20)=0x50\text{Device address} = 0b1010 \, 0000 + (0 \times 2^2 + 0 \times 2^1 + 0 \times 2^0) = 0x50

The connection between Raspberry Pi Pico and the external EEPROM module (AT24C256 chip) is shown in the following table:

device

Module Signals / Pins

Pico Pin (GPIO)

AT24C256 chip

SDA

GP2

AT24C256 chip

SCL

GP3

Meanwhile, the AT24CXX class that we defined ourselves when explaining the I2C protocol will also be placed in a separate Python file at24cxx.py, the sample code is as follows:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/27 PM 10:42
# @Author  : Li Qingshui
# @File    : at24cxx.py       
# @Description : Implements an AT24CXX class for operating AT24CXX series EEPROMs

# ======================================== Import Modules =========================================

# Hardware-related modules
from machine import I2C, Pin
# Time-related modules
import time

# ======================================== Global Variables ============================================

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

class AT24CXX:
    """
    AT24CXX class, used to operate AT24CXX series EEPROM chips over the I2C bus, supporting EEPROMs of multiple capacities.

    This class encapsulates I2C communication with the AT24CXX EEPROM and provides functions such as byte and page writing, reading, and erasing data.
    Supports EEPROM chips of different capacities (from 32KiB to 64KiB), reading and writing data over I2C.

    Attributes:
        i2c (I2C): I2C instance used to communicate with the AT24CXX chip.
        chip_size (int): Size of the EEPROM chip, default 64KiB.
        addr (int): I2C device address, default value is 0x50.
        max_address (int): Maximum accessible address of the device, depending on the size of the EEPROM.

    Methods:
        __init__(self, i2c, chip_size=AT24C512, addr=0x50):
            Initialize an AT24CXX class instance.

        write_byte(self, address: int, data: int) -> None:
            Write one byte to the specified address.

        read_byte(self, address: int) -> int:
            Read one byte from the specified address.

        write_page(self, address: int, data: bytes) -> None:
            Write one page of data to the specified address.

        read_sequence(self, start_address: int, length: int) -> bytes:
            Read data of the specified length sequentially.
    """

    # Different capacities of the AT24CXX series EEPROM
    AT24C32 = 4096      # 4KiB
    AT24C64 = 8192      # 8KiB
    AT24C128 = 16384    # 16KiB
    AT24C256 = 32768    # 32KiB
    AT24C512 = 65536    # 64KiB

    def __init__(self, i2c, chip_size: int = AT24C512, addr: int = 0x50) -> None:
        """
        Initialize an AT24CXX class instance.

        Args:
            i2c (I2C): I2C instance used to communicate with the AT24CXX chip.
            chip_size (int): Size of the EEPROM chip, default is AT24C512.
            addr (int): I2C device address, default is 0x50.

        Raises:
            ValueError: If the provided chip_size is not within the supported range.
        """
        # Check whether the EEPROM chip capacity is within the range defined by the AT24CXX class
        if chip_size not in [AT24CXX.AT24C32, AT24CXX.AT24C64, AT24CXX.AT24C128,
                             AT24CXX.AT24C256, AT24CXX.AT24C512]:
            raise ValueError("chip_size is not in the range of AT24CXX")

        self.i2c = i2c
        self.chip_size = chip_size
        self.addr = addr
        # Maximum address of the chip that the user can access
        self.max_address = chip_size - 1

    def write_byte(self, address: int, data: int) -> None:
        """
        Write one byte to the specified address.

        Args:
            address (int): Address to write to.
            data (int): Data to write, range 0-255.

        Raises:
            ValueError: If the address or data is out of the valid range.
        """
        # Check whether the address is within the valid range
        if address < 0 or address > self.max_address:
            raise ValueError('address is out of range')

        # Check whether the data is within the valid range
        if data < 0 or data > 255:
            raise ValueError("data must be 0-255")

        # Starting from the user-specified memory address `address`, write bytes([data]) to the EEPROM at device address addr
        # The memory address is 16 bits, two bytes
        self.i2c.writeto_mem(self.addr, address, bytes([data]), addrsize=16)
        # Delay 5 ms to wait for the EEPROM write to complete
        time.sleep_ms(5)

    def read_byte(self, address: int) -> int:
        """
        Read one byte from the specified address.

        Args:
            address (int): Address to read from.

        Returns:
            int: The data read.

        Raises:
            ValueError: If the address is out of the valid range.
        """
        # Check whether the address is within the valid range
        if address < 0 or address > self.max_address:
            raise ValueError("address is out of range")

        # Read one byte from the specified address
        value_read = self.i2c.readfrom_mem(self.addr, address, 1, addrsize=16)
        # Convert to an integer and return, using big-endian conversion
        return int.from_bytes(value_read, "big")

    def write_page(self, address: int, data: bytes) -> None:
        """
        Write one page of data to the specified address, handling page-crossing cases.

        Args:
            address (int): Starting address to write to.
            data (bytes): Data to write, with no limit on maximum length.

        Raises:
            ValueError: If the address, data length, or data is out of the valid range.
        """
        # Check whether the address is within the valid range
        if address < 0 or address > self.max_address:
            raise ValueError("address is out of range")

        # Check whether any data in the list is out of range
        for i in data:
            if i < 0 or i > 255:
                raise ValueError("data must be 0-255")

        # Check whether the length of data, combined with the starting address, is out of range
        if address + len(data) > self.max_address:
            raise ValueError("data exceeds maximum limit")

        # Get the boundary of the starting page
        page_boundary = (address // 64 + 1) * 64

        # Write the data in segments
        while data:
            # Calculate the number of bytes to write this time
            write_length = min(len(data), page_boundary - address)

            # Write one page of data to the specified address
            self.i2c.writeto_mem(self.addr, address, data[:write_length], addrsize=16)
            # Delay after writing to ensure completion
            time.sleep_ms(5)

            # Update the address and data
            address += write_length
            data = data[write_length:]

            # Update the page boundary
            page_boundary = (address // 64 + 1) * 64

            # If the current address exceeds the maximum address, stop writing
            if address > self.max_address:
                raise ValueError("address exceeds maximum limit")

    def read_sequence(self, start_address: int, length: int) -> bytes:
        """
        Read data of the specified length sequentially; the AT24CXX can read across pages and is not limited by page boundaries.

        Args:
            start_address (int): Starting address to read from.
            length (int): Number of bytes to read.

        Returns:
            bytes: The data read.

        Raises:
            ValueError: If the starting address and length are out of the valid range.
        """
        # Check whether the starting address and length are within the valid range
        if start_address < 0 or (start_address + length) > self.max_address:
            raise ValueError("address is out of range")

        # Read data of the specified length from the specified starting address
        return self.i2c.readfrom_mem(self.addr, start_address, length, addrsize=16)

# ======================================== Initialization ==========================================

# ========================================  Main Program  ===========================================

3.5.2 Implementation of EEPROM Block Device Class

Here, we first define a EEPROMBlockDevice class to manage EEPROM (e. g., AT24C256) as a block device, providing block read/write operations and device control functions. This class, like the RAM block device RAMBlockDev class, inherits from the abstract base class AbstractBlockDev and implements the functions of reading, writing, controlling, and erasing blocks.

The sample code is shown below:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/27 PM 10:45
# @Author  : Li Qingshui
# @File    : eeprom_block_dev.py       
# @Description : Defines an EEPROMBlockDevice class, including block read/write operations and device control functions

# ======================================== Import Modules =========================================

# Import the abstract base class of the virtual file block device
from AbstractBlockDevInterface import AbstractBlockDev
# Import the AT24CXX class
from at24cxx import AT24CXX

# ======================================== Global Variables ============================================

# Define error code constants, conforming to POSIX standard error codes
ENOERR = 0       # Operation successful
EPERM = 1        # Operation not permitted
EIO = 5          # I/O error
ENODEV = 19      # Invalid device/block number
EROFS = 30       # Read-only file system
EINVAL = 22      # Invalid argument
ENOSPC = 28      # No space

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

# Custom EEPROM block device class, inherits from the abstract block device class
class EEPROMBlockDevice(AbstractBlockDev):
    """
    EEPROM block device implementation class, providing a block device interface that meets MicroPython file system requirements.

    This class implements the AbstractBlockDev abstract interface and supports emulating a block device through EEPROM memory,
    and can be used to mount MicroPython file systems (such as FAT, LittleFS, etc.).

    Attributes:
        eeprom (AT24CXX): AT24CXX series EEPROM driver instance
        block_size (int): Block size in bytes; must be an integer multiple of the EEPROM page size
        _is_initialized (bool): Device initialization state flag

    Methods:
        __init__(self, eeprom, block_size=512):
            Initialize the EEPROM block device instance
        readblocks(self, block_num, buf, offset=0):
            Read data from the specified block into a buffer
        writeblocks(self, block_num, buf, offset=0):
            Write data to the specified block or erase the block
        ioctl(self, op, arg):
            Device control operations
        _validate_block_num(self, block_num):
            Validate the block number (internal method)

    Note:
        - The block size must be an integer multiple of the EEPROM page size (usually a multiple of 8)
        - The block must be erased (0xFF) before a write operation
        - All methods are thread-safe
    """
    def __init__(self, eeprom: AT24CXX, block_size: int = 512) -> None:
        """
        Initialize the EEPROM block device instance.

        Args:
            eeprom (AT24CXX): AT24CXX series EEPROM driver instance
            block_size (int, optional): Block size (in bytes); must be aligned with the file system block size. Defaults to 512,
                                        and must also be an integer multiple of the EEPROM page size.

        Returns:
            None

        Raises:
            ValueError (EINVAL): If the block size does not meet the requirements
            OSError (ENODEV): If the EEPROM device is invalid
        """
        # Check whether the EEPROM page size is an integer multiple of the block size
        # The page size of AT24CXX series EEPROMs is an integer multiple of 8 bytes, so the block size must be an integer multiple of 8
        if block_size % 8 != 0 or block_size < 8:
            raise ValueError(EINVAL, "Block size must be multiple of 8")

        # Check whether the passed-in eeprom is an AT24CXX series EEPROM
        if not isinstance(eeprom, AT24CXX):
            raise OSError(ENODEV, "Invalid EEPROM device")

        # Parent class initialization function
        super().__init__()
        # Save the EEPROM instance
        self.eeprom = eeprom
        # Set the block size
        self.block_size = block_size
        # Initialize the device state to not enabled
        self.device_open = False

    def readblocks(self, block_num: int, buf: bytearray, offset: int = 0) -> None:
        """
        Read data from the specified block into a buffer.

        Args:
            block_num (int): Starting block number (starting from 0).
            buf (bytearray): Destination buffer; its length must be <= the block size.
            offset (int, optional): Offset within the block (not yet implemented). Defaults to 0.

        Returns:
            None

        Raises:
            ValueError (EINVAL): If offset≠0 or the buf length is invalid
            OSError (EIO): If the read fails
            OSError (ENODEV): If the block number is invalid
        """
        # Check whether the offset is 0
        if offset != 0:
            raise ValueError(EINVAL, "Offset must be 0")

        # Check whether the buffer length exceeds the block size
        if len(buf) > self.block_size:
            raise ValueError(EINVAL, "Buffer length exceeds block size")

        # Check whether the block number is valid
        if block_num < 0 or block_num >= self.ioctl(4, 0):
            raise OSError(ENODEV, "Invalid block number")

        # Read data
        try:
            # Calculate the starting address for writing
            addr = block_num * self.block_size
            # Write the data that was read into the buffer
            buf[:] = self.eeprom.read_sequence(addr, len(buf))
        except Exception as e:
            # Print the error message
            raise OSError(EIO, "Read failed") from e

    def writeblocks(self, block_num: int, buf: bytearray | None, offset: int = 0) -> None:
        """
        Write data to the specified block or erase the block.

        Args:
            block_num (int): Target block number (starting from 0).
            buf (bytearray | None): Data to write (None means erase the block).
            offset (int, optional): Offset within the block (not yet implemented). Defaults to 0.

        Returns:
            None

        Raises:
            ValueError (EINVAL): If the parameters are invalid
            OSError (EIO): Write failed
            OSError (ENOSPC): Block number out of range
        """
        # Check whether the offset is 0
        if offset != 0:
            raise ValueError(EINVAL, "Offset must be 0")

        # Calculate the length of data to write
        length = len(buf)
        # Check the data read length
        if length > self.block_size:
            raise ValueError(EINVAL, "Buffer too large")

        # Check whether the block number is valid
        if block_num < 0 or block_num >= self.ioctl(4, 0):
            raise OSError(ENOSPC, "Block number out of range")

        # Try to write the data
        try:
            # Calculate the starting address for writing
            addr = block_num * self.block_size
            # If buf is None, erase the block
            if buf is None:
                self.eeprom.write_page(addr, bytearray([0xFF]*self.block_size))
            else:
                # Write data
                self.eeprom.write_page(addr, buf)
        except Exception as e:
            raise OSError(EIO, "Write failed") from e

    def ioctl(self, op: int, arg: int) -> int | None:
        """
        Device control operation.

        Args:
            op (int): Operation code (use AbstractBlockDev constants):
                - IOCTL_INIT (1): Initialize the device
                - IOCTL_SHUTDOWN (2): Shut down the device
                - IOCTL_SYNC (3): Synchronize data
                - IOCTL_BLK_COUNT (4): Get the total number of blocks
                - IOCTL_BLK_SIZE (5): Get the block size
                - IOCTL_BLK_ERASE (6): Erase the specified block
            arg (int): Operation argument (depends on op):
                - IOCTL_BLK_ERASE: Block number to erase
                - Can be ignored for other operations

        Returns:
            int | None:
                - IOCTL_BLK_COUNT: Returns the total number of blocks
                - IOCTL_BLK_SIZE: Returns the block size
                - Other successful operations return 0
                - Unsupported op returns None

        Raises:
            ValueError: If the block number to erase is invalid
            NotImplementedError: If the operation code is not supported
        """
        # Initialize the device
        if op == AbstractBlockDev.IOCTL_INIT:
            # Set the device state to enabled
            self.device_open = True
            # Return success
            return ENOERR
        # Shut down the device
        elif op == AbstractBlockDev.IOCTL_SHUTDOWN:
            # Set the device state to disabled
            self.device_open = False
            # Return success
            return ENOERR
        # Synchronize the device
        elif op == AbstractBlockDev.IOCTL_SYNC:
            # Perform the synchronization operation (if needed)
            # Return success
            return ENOERR
        # Get the number of blocks
        elif op == AbstractBlockDev.IOCTL_BLK_COUNT:
            # Calculate and return the number of blocks
            return self.eeprom.chip_size // self.block_size
        # Get the block size
        elif op == AbstractBlockDev.IOCTL_BLK_SIZE:
            # Return the block size
            return self.block_size
        # Erase the specified block
        elif op == AbstractBlockDev.IOCTL_BLK_ERASE:
            # Get the block number to erase
            block_num = arg

            # Check whether the block number is valid
            if block_num < 0 or block_num >= self.eeprom.chip_size // self.block_size:
                raise ValueError(EINVAL, "Invalid block number")
            # Erase the specified block
            self.writeblocks(arg, None)
            # Return success
            return ENOERR
        else:
            # Unsupported operation code
            raise NotImplementedError("Unsupported ioctl operation")

# ======================================== Initialization ==========================================

# ========================================  Main Program  ===========================================

Here, the EEPROMBlockDevice class has the following methods:

20 File System and Storage Architecture - Draft Document Tutorial
20 File System and Storage Architecture - Draft Document Tutorial
20 File System and Storage Architecture - Draft Document Tutorial
20 File System and Storage Architecture - Draft Document Tutorial

3.5.3 Application Experiment

The following code can be found in the elegance-devkit v1\Demo\74 FileSys_EEPROM folder within the resource package we provide.

In the main program, we create and mount a FAT file system on the AT24C256 EEPROM chip to store and read file data. Meanwhile, we test the data read/write function of the file system on the EEPROM chip, as well as the power-off data retention function of the EEPROM chip.

The sample code is as follows:

# Python env   : MicroPython v1.23.0
# -*- coding: utf-8 -*-        
# @Time    : 2024/9/27 10:43 PM
# @Author  : Li Qingshui
# @File    : mian.py       
# @Description : Used with the virtual file system class; mount a file system on an external EEPROM chip

# ======================================== Import Modules =========================================

# Hardware-related modules
from machine import I2C, Pin
# Time-related modules
import time
# Import the AT24CXX class
from at24cxx import AT24CXX
# Import the custom EEPROM block device class
from eeprom_block_dev import EEPROMBlockDevice
# Import the virtual file class
import vfs
# Import modules related to file operations
import os
# Import the JSON module
import json

# ======================================== Global Variables ============================================

# The AT24C256 chip address is 0x50, i.e. 0b1010000
# The upper 7 bits are the address bits and the lowest bit is the read/write control bit: 0 for write, 1 for read
AT24C256_ADDRESS = 0x50

# ======================================== Functions ============================================

# ======================================== Custom Classes ============================================

# ======================================== Initialization ==========================================

# Power-on delay of 3 s
time.sleep(3)
# Print debug message
print("FreakStudio: Mount the external EEPROM to the file system")

# Create a hardware I2C instance using the I2C0 peripheral, with a clock frequency of 400KHz, SDA pin 4, and SCL pin 5
i2c_at24c256 = I2C(id=0, sda=Pin(4), scl=Pin(5), freq=400000)

# Create an AT24C256 instance
at24c256 = AT24CXX(i2c_at24c256, AT24CXX.AT24C256, AT24C256_ADDRESS)

# Create the block device using the external EEPROM chip, with a block size of 512 bytes and 64 blocks in total
block_device = EEPROMBlockDevice(eeprom=at24c256, block_size=512)

# Create a FAT file system on the block device
vfs.VfsFat.mkfs(block_device)
# Mount the block device to the /eeprom directory of the virtual file system
vfs.mount(block_device, '/eeprom')

# ========================================  Main Program  ===========================================

# Use the os module to list the current directory and print it
print("Directory contents :",os.listdir())

# Configuration data
config_data = {
    "micropython_version": "v1.23.0",
    "required_modules": ["machine", "time", "vfs", "os", "json"],
    "author": "leeqingshui",
    "company": "FreakStudio"
}
# Convert the configuration data to JSON format
json_data = json.dumps(config_data)

# Create the configuration file and write the data
with open('/eeprom/config.json', 'w') as file:
    # Write the JSON data
    file.write(json_data)
# Print debug information
print("Config file created.")

# Read the file
try:
    # Open the configuration file for reading
    with open('/eeprom/config.json', 'r') as file:
        # Read the file contents
        content = file.read()
        # Print the file contents
        print("Config file content:")
        print(content)
# Catch file operation exceptions
except OSError as e:
    # Print the error message
    print("Error reading file:", e)

# Monitor the EEPROM power state; here we will unplug the EEPROM power
# and then reconnect it later to test whether data is retained after power loss
print("Monitoring EEPROM power...")

# Infinite loop to determine whether the EEPROM has lost power
while True:
    try:
        # Try to scan I2C devices
        devices = i2c_at24c256.scan()
        # Check whether the EEPROM address is in the device address list
        if AT24C256_ADDRESS in devices:
            print("EEPROM is powered on.Waiting...")
        else:
            # If it is not in the device address list, the EEPROM has lost power
            print("EEPROM is powered off.")
            # Exit the loop
            break
        # Wait 1 second before checking again
        time.sleep(1)
    # Catch the exception
    except Exception as e:
        # Print the scan error message
        print("Error scanning I2C devices:", e)
        # Wait 1 second before trying again
        time.sleep(1)

# Infinite loop to read the EEPROM data after power-up
while True:
    # Scan I2C devices
    devices = i2c_at24c256.scan()

    # Check whether the EEPROM address is in the device address list
    if AT24C256_ADDRESS in devices:
        # Print the message that the EEPROM has been powered up again
        print("EEPROM is powered on.")
        # Read the configuration file
        try:
            # Open the configuration file for reading
            with open('/eeprom/config.json', 'r') as file:
                # Read the file contents
                content = file.read()
                # Print the file contents
                print("Config file content after power restoration:")
                print(content)
        # Catch file operation exceptions
        except OSError as e:
            print("Error reading file after power restoration:", e)
        # Exit the loop
        break
    # Wait 1 second before continuing to scan
    time.sleep(1)

Here, we have carried out the following work:

20 File System and Storage Architecture - Draft Document Tutorial
  1. Use the defined AT24C256 instance to create a block device, and set the block size to 512 bytes.
  2. Format the block device as a FAT file system and mount it to the /eeprom directory of the virtual file system.
  3. Define a piece of configuration data via a dictionary, convert it to JSON format, and then write it to the /eeprom/config.json file.
  4. Open the configuration file, read it, and print the file content.
  5. Enter an infinite loop to continuously check the power status of the EEPROM, and exit the loop if the EEPROM loses power.
  6. Enter another infinite loop to continuously scan for I2C devices. If it is detected that the EEPROM has been powered back on, attempt to open the configuration file, read its content, and then terminate the program.

Burn the code, open the terminal, and the following content will be displayed:

20 File System and Storage Architecture - Draft Document Tutorial

When the terminal outputs in a loop EEPROM is powered on.Waiting.. ., we can unplug the Fengya No. 1 Board - Environment and Storage Acquisition Board to power it off; after waiting for a while, plug in the Fengya No. 1 Board - Environment and Storage Acquisition Board again to power it on.

It can be seen that the EEPROM chip can normally implement the data read/write function of the file system and the power-off data retention function.

3.5 Application Experiment.mp4 13.7 MB

We can use the following command to test whether the erase function works properly:

os.remove('eeprom/config.json')

The above command has deleted the config.json file. Then enter the following command to view the files in the eeprom folder:

os.listdir('eeprom')

It can be seen that the file has been deleted at this point:

20 File System and Storage Architecture - Draft Document Tutorial

References

https://docs.singtown.com/micropython/zh/latest/openmvcam/reference/filesystem.html

https://pico.nxez.com/2023/10/23/data-logger.html

https://www.renesas.com/en/support/engineer-school/mcu-programming-peripherals-05

https://www.cnblogs.com/sunshine-jackie/p/8137377.html

https://docs.micropython.org/en/latest/library/vfs.html#vfs.AbstractBlockDev

https://www.cnblogs.com/Javaer1995/p/18331745

https://www.sd-nand.com/news/technology/277.html

https://www.eeworld.com.cn/mcu/eic666379.html

https://blog.csdn.net/yuanzywhu/article/details/131592815

https://blog.csdn.net/weixin_37366441/article/details/120461015

https://cloud.tencent.com/developer/article/2179371

Addendum: closure

A closure is an important concept in Python. It refers to a function defined inside another function, and this inner function can access the variables of the outer function even after the outer function's scope has terminated; simply put, a closure is a function that retains access to the variables in the outer scope where it was defined, even when those variables no longer exist in that outer scope. This feature allows us to create more flexible functions, such as returning functions with specific behaviors without explicitly passing all required parameters.

Three conditions for closure:

In the following code, we implement a closure inner_function:

def outer_function(x):
    def inner_function(y):
        # inner_function can access the x variable of outer_function
        return x + y  
    return inner_function

In this example:

When using closures, we need to note that the execution of a closure can be divided into the following stages:

  1. Definition phase: When an outer function is invoked, the inner function (i. e., a closure) is defined and captures the variables within the outer function; at this point, the outer function returns the closure function, but the closure has not yet been executed
  2. Invocation Phase: The closure will only be executed when you explicitly call the returned closure function; when the closure is executed, it will use the captured external variables and perform operations or other actions as needed

For example, for the closure inner_function implemented above, using the following code will only call the outer function, not the closure:

# outer_function executes and returns the closure, which captured x=5, but the closure has not been executed yet
closure = outer_function(5)  

When we call closure (3), the closure will execute, using the previously captured x= 5 and the currently passed-in y= 3 for calculation, and return the result:

# Now we can call the closure, passing in y=3, and the closure starts executing
result = closure(3)
# Outputs 8
print(result)  

Using closure has the following advantages:

Version Record

Version Number

Modifier

Time

Content

v1.2.0

Li Zisheng

2025/12/24

Initialize the document, based on MicroPython version v1.23.0.