Add offline voice control without making users learn the wrong commands

You want visitors to speak to a small interactive object: start the light, stop it, make it blink. An offline voice module looks like the quickest route. But a long list of supported commands does not help much if none of them are the words people would naturally say to your object.

That gap shows up clearly in a DFRobot forum discussion from April 2023. An escape-room prop builder wanted a handful of everyday responses and numbers, while the module's preset phrases were aimed at other uses. So the first question is small: can this prototype recognize the few phrases your interaction needs, deliver the right command, and show that it was carried out?

On our bench we tested that with a GraftSense ASRPRO-CORE voice module, a speaker and a GraftPort controller board with a Raspberry Pi Pico running MicroPython. The voice module sends one of three command bytes; the controller blinks, turns off or turns on an LED.

Give the voice module and the controller different jobs

The voice module listens for configured phrases. The controller receives a short serial command and decides what to do. That split lets you test recognition and physical behavior separately: first check which byte arrived, then check whether the output changed.

The carrier board combines the ASRPRO-CORE module, a microphone, a speaker connector, a host serial connector and a USB programming interface. USB programs the voice side; the host connector carries commands during operation. They are different paths even though both are serial.

The host connector is labeled GND, VCC, MTX and MRX. The M refers to the main controller: the controller's transmit line goes to the module's receive line and vice versa. Do not apply a generic "TX to TX" rule from printed names alone.

We ran the link at 115200 baud and drove the LED on GPIO25. These assignments describe our setup; check the board you actually use. We used an 8-ohm, 2-watt speaker and powered the UART side before plugging in the USB programming cable. Treat that power sequence as specific to this carrier. The core's supply range and its serial signal levels are separate questions, and neither can be read from the connector's shape.

Configure the words and the bytes together

There are two programs to keep in step: the voice project on the module and main.py on the controller. The voice project, built in the module's graphical editor, holds the wake phrase, the recognition entries, the spoken replies and the serial output settings. Change the pair together.

Our test project used Mandarin phrases. Nothing in this test establishes English recognition; for English commands you need to build and validate an English voice project.

These are single byte values, not the two text characters 0 and 1. An editor set to send the text 01 produces a different sequence from the byte 0x01, and the receiver will not match it.

The module's general reference mentions a 9600-baud default, while our project explicitly used 115200 on both sides. Match the loaded project, not a default copied from another example. Also keep the UART numbers apart: the module sends on its own UART1 and the controller receives on its UART0. The numbering belongs to each device, not to the cable.

Watch the output, not just the spoken reply

The receiver sets up the UART and reads one byte at a time:

uart = machine.UART(0, baudrate=115200,
                    tx=machine.Pin(16), rx=machine.Pin(17))
if uart.any():
    cmd = ord(uart.read(1))

It then compares cmd with the three values in the table and changes the LED. A blink is two cycles of half a second on and half a second off. The MicroPython UART reference covers the receive methods.

Our recording shows the LED changing on the connected hardware. Keep a serial log next to that visual check when you reproduce it; silent video frames cannot show what was said or measure recognition reliability.

The voice module also plays replies meaning the light has blinked, switched off or switched on. Those replies are configured on the voice module. The controller program does not send an acknowledgement back, so hearing the reply does not prove the controller changed its output. For a prop with a visible LED that may be enough during a first test. A hidden mechanism needs a real completion signal.

Try one more interaction early: say "off" while a blink is running. The blink routine blocks the main loop for about two seconds, so the next command waits until it returns. If stopping promptly matters, replace the blocking blink with a timed state machine that keeps reading commands. That is the next development step; we have not built it yet.

Try the intended users and the intended enclosure

Once the three commands work on the bench, choose the words with the people who will use the object. A museum visitor, a workshop participant or a performer may phrase the same action differently from its designer. Record the spoken phrase, the byte received, the visible action and whether another attempt was needed. Include different speakers and the background sound expected in the real setting.

Then repeat the checks in a provisional enclosure. Keep the programming connector reachable, and route the microphone cable so it is not strained every time the cover opens. Integration guidance for the core covers stable supply with local decoupling, short and protected microphone routing, short and wide speaker routing, and keeping the module's UART0 free for firmware updates. Use that as a checklist, not as a measured result. The illustrations we had for this carrier disagree on speaker pin numbering, so confirm polarity from the carrier schematic and the physical connector before making a custom harness.

A recent Home Assistant discussion asks for compact, finished voice hardware with good microphones and speakers rather than another stack of development boards. That is one user's requirement, not evidence this module meets it, but it points to the right test: can someone use the finished-shaped object comfortably from where they will actually stand?

Do not quote the module's advertised recognition percentage or response time as your answer. Those figures come without the phrase set, speakers, distance, noise conditions and trial count needed to reproduce them. A small test record for your actual interaction is more useful.

Decide what the next prototype should prove

This setup gives a manageable first milestone: a configured phrase becomes a known byte, and that byte produces a visible action. Keep a manual way to operate the object while testing voice, and make missed commands easy to notice. Then decide whether fixed phrases are enough or whether the project needs a more flexible speech system.

The question for the next build is no longer "can we add voice?" It is "what must the person say, hear and see for this object to work?"

Related service: Hardware engineering & prototype · Related build: Offline voice-controlled robot dog · More on displays, controls and feedback