For a music workshop or interactive exhibit, showing an object to a sensor and hearing a tone can be an inviting first action. Adafruit's PianoGlove, first published in 2013, is one example of using a color sensor to make music. For a device shared by visitors there is another design question: how does someone know when it is collecting colors, when it is playing, and how to start again?
On our bench we built a small color-and-sound prototype to make that question concrete. It combines a TCS34725 color sensor, a capacitive touch input and an LM386-based speaker module on a GraftPort-Pico controller, mounted in a clear acrylic enclosure. Holding the touch input collects color samples; releasing it starts playback. Touching again interrupts playback and begins a new collection. That sequence gives a first prototype a beginning, an end and a way to recover from a mistaken action.
Choose the interaction before polishing the sound
For a first demonstration, the firmware maps the strongest RGB channel to a tone: red to 500 Hz, green to 1000 Hz and blue to 1500 Hz, with a 2000 Hz fallback when no channel wins. RGB is simply the separate red, green and blue contribution measured by the sensor. The mapping makes changes audible without a screen.
These frequencies are an authored interaction rule, not a natural relationship between colors and musical notes. A workshop activity might benefit from that obvious difference; an instrument would need a more deliberate musical mapping. Keep the simple rule while you find out whether people understand the action.
The touch module carries the visitor's gesture. The controller's onboard button does something else: it pauses and resumes the whole task. Those two controls are not interchangeable, and a visitor-facing build should make clear which one they are meant to use.
Connect sensing, input and playback separately
The color sensor plugs into the I2C0 connector, the speaker module into DIO1 and the touch module into DIO0. That separation helps when the prototype does not respond: first check whether the touch is detected, then whether color data changes, then whether the speaker produces the commanded output.
Before closing the enclosure, leave room to place an object consistently in front of the sensor and to touch the input without disturbing the cables. We mounted the boards on M3 standoffs inside acrylic panels, which made those distances easy to see and adjust.
Keep the controls responsive during playback
The application task tracks whether the input is pressed, whether sampling is active, the recorded sequence and whether playback is running. It runs every 200 ms. Playback runs separately, so playing a sequence does not occupy the same loop that checks for another touch.
One application task coordinates three drivers: color in, touch in, sound out. That organization answers the original question: pressing again must stay meaningful while the device is busy producing sound. It also means the firmware has to coordinate shared state carefully. Calling playback "asynchronous" describes the design, not a measured guarantee of instant interruption.
The configuration retries I2C initialization up to three times with a half-second delay, keeps debug output on so sampling and playback messages are visible, and starts the task automatically.
Test a complete visitor action
Our test presented differently colored objects to the sensor, checked the RGB output, released the touch input to play the sequence, and touched again during playback. Keep those actions together. Hearing one tone establishes much less than completing the whole collect, play and restart loop.
The debug log showed that loop working. It recorded each collection start, the samples with their assigned color and frequency, the number of notes played, and, in one run, a touch during playback that stopped the sound and started a new collection.
The same log also shows the weak point of a first build. The raw channel values were single-digit counts, for example R=1, G=0, B=0 classified as red, and several samples came out as R=0, G=1, B=1, which fell through to the 2000 Hz fallback. When the winning channel leads by one count, the classification depends on lighting, distance and noise more than on the object's color. That is a sensing and illumination problem to solve before tuning the sound.
We made no acoustic measurement; the frequencies above are what the firmware commanded, not measured pitch or loudness.
For the next user trial, use the same objects and ask someone unfamiliar with the build to make a short sequence, stop it and repeat it. Watch where they hesitate. If they cannot tell whether it is collecting or playing, add an explicit state indicator before redesigning the enclosure.
Make the sensing conditions part of the design
A color input includes the lighting and the object's position, not just the sensor model. Start with repeatable placement and controlled lighting close to the sensor, then change them deliberately. An exhibit in a bright window and the same exhibit under colored room lighting may need different handling of uncertain samples.
Decide what the device should do when no color clearly dominates, rather than silently treating every sample as a confident classification. A minimum signal level, a margin between the top two channels, or a distinct "not sure" sound are all options to test. The useful next milestone is a repeatable interaction that someone can discover and control without being coached.