Summer of Sound: What I learned by trying to build a musical instrument out of 3D printer electronics.

Musical instruments have often frustrated me and I wanted to see if I could take the electronic guts from a 3D printer and make something better – or at least something that was fun in a Rube Goldberg sort of way! This is the story of that project, what works, what still needs improvement and what I learned along the way.

The idea was simple. Playing a stringed instrument is a matter of learning to get the strings to vibrate at the right frequencies at the right times. On a guitar, you do so by pressing down with your fingers on the fingerboard, thus changing the length of the portion of the string which is allowed to vibrate. The challenge comes from committing to muscle memory the various hand positions for each chord and knowing how to quickly and accurately move from one another. All of this is quite tedious and is ripe for automation!

A consumer-grade 3D printer contains several stepper motors, a power supply and a controller board. These motors are quite precise, in the typical case allowing the shaft to be rotated to positions that are accurate within 1.8 degrees. Rather than using a fingerboard, I wanted to change the pitches in my instrument by use the stepper motors to vary the tension on the nylon strings.

For my prototype, I used a few half-height NEMA-17 stepper motors, a 3D printer controller board and a 24V power supply. I designed a neck piece for mounting the motors, bridge and tuning pegs in FreeCAD and printed those out on a 3D printer. Everything was mounted to a board and I used inexpensive mono-filament fishing line for the strings.

The controller board is an unmodified board from a 3D printer. While I could write my own firmware for it, I decided to jump start my project by using Marlin, an open-source 3D printer firmware. The board would accept commands over serial port in GCODE, just as a 3D printer would, but I would be using it just for the motion commands. The two changes I needed to do to Marlin was modify the configuration file so that Marlin would not issue a temperature error when I left the temperature sensors open. I also changed the endstop configuration to normally open, so I could leave the endstops disconnected.

For the client software, I decided to write a GUI in Python. The GUI had two major functions. It would provide interactive control of the stepper motors, as well as do the audio processing necessary for tuning the strings. For my GUI, I chose to use the tkinter toolkit, since it is available on all versions of Python.

Since I did not know how to do audio processing in Python, I searched the web for a starting point. It turned out that Matt Zucker had written a very basic command-line Ukulele tuner in Python. His code gave me a good start. It uses the pyaudio library to load audio samples into a buffer, applied a Hanning window, and ran the result through numpy's fast-fourier transform function for real valued inputs.

The tuner would print note names on the text console, but it would often print note names even when everything was silent. I improved it by using the peakutils library to isolate peaks and added a threshold to detect the start and end of plucked notes. I also used mathplotlib to show a window with a real-time graph of the spectrum data.

The second part of the GUI allows the user to select a serial port and send serial commands to Marlin. This was accomplished using the pyserial library. Although Marlin has a commands for all sorts of things, I was only interested in commanding the stepped motors. The following commands are the most useful:

Full documentation on GCODE is provided in the Marlin website.

My goal is to make it so the software can automatically change from chord to chord by adjusting string tensions, but it turns out this is a much harder problem to solve than I had expected. There are several uncertainties which make this a difficult problem. The pitch detection data is susceptible to noise and false reading. I also found out that the strings have quite a bit of stretch to them, making it very difficult to return to the precise tension needed for one pitch when transitioning from another.

Although I found there is a rough linear relationship between a string's fundamental vibrating frequency and the stepper position, this relationship is not perfect and when making a large jump from one note to another, one often ends up a semi-tone or two off pitch.

Right now I suspect it is impossible to land a pitch precisely in one go and that closed-loop feedback will be necessary to quickly re-tune a string just after it is plucked. I envision it working something like this:

At the moment, I have a rough algorithm for auto-tuning of one string at a time, but I am working on making it reliable enough to tune all strings at once.

All of the software I have written for this project is open-source and is available on github.