Skip to main content

Student Innovation - Drumless: Drum More

transparent1_ab38a98c6df8c77ccab26c5d50c3f028410ea984.png

I used to play in a rock band at school and after several months at university, I really started to miss playing the guitar. However, I knew it was impractical to bring any musical instrument to my tiny dorm room. After speaking to a few of my university friends I realised that this is a common problem among students, so I decided to apply my fresh knowledge in electronics engineering and do something about it. I wanted to reproduce the experience of playing a real musical instrument without the irritating inconvenience of transportation, storage and potentially angry neighbours. After a week of brainstorming and some inspiration from the calculus class, I decided that I can make an air-drumming device. But before I explain how, I would like to show a quick two-minute demo of the final product.

I have to be honest, completing the full project from idea to what you saw in that video took me two and a half years. Yet, the concept is really simple and I will summarise it the following paragraphs and hopefully inspire you to build something yourself.

Hardware

To start with, let's take a look at Figure 1 and see what's inside a drumstick. The main parts that are relevant to this article are:

  • Sensors - A collection of an accelerometer, gyroscope and a magnetometer. The accelerometer and the gyroscope come in a single chip ICM-20602 by InvenSense, whilst the magnetometer is LIS2MDL by ST.
  • MCU - The microcontroller unit (MCU) is nRF52832 by Nordic Semiconductor and is built around Arm® Cortexâ„¢-M4 CPU with floating point unit running at 64 MHz. The microcontroller supports Bluetooth Low Energy (BLE), which make communication with a mobile device trivial.

The rest of the hardware is power management, radio, user inputs and light indication.

POWERPNT_Gvk8FvRVTE_488b84d9f122e14a5156f7a178f7b1c90459f2fb.png

Figure 1. The inside of a drumstick.

The MCU fetches motion data from the sensors via SPI. It then processes the motion data and once a drum strike is detected, a BLE MIDI message is sent to a mobile phone which plays the drum sound through some music app (e.g. GarageBand). BLE MIDI is a standard protocol when it comes to digital music instruments, so Drumless is actually supported by most music apps.

Software

Just by looking at the hardware, it is probably quite obvious that the data from the sensors is used to trigger the drum sounds. In this section, I will break the problem of air-drumming down into three sub-problems - strike detection, velocity detection, and drum detection.

Strike Detection

The strike detector is a function that takes as an input the motion data from the sensors and outputs true if the motion data is likely to be a drum strike and false otherwise. If the output is true, then a BLE MIDI message is sent to a music application on the mobile phone and a sound is played.

To understand how this function works, let's take a look at Figure 2. The position of the tip of the drumstick is tracked over time as the drumstick is moved in a sinusoidal motion. Unsurprisingly, this motion resembles the motion of a drumstick while playing the drums. If you take a closer look, you would notice that each drum strike would correspond to a valley (the opposite of a peak) in the tracked position.

Figure 2. The trail left by the tip of the drumstick, while moving it in a sinusoidal motion. (Hand infographic vector created by pch.vector - www.freepik.com)

Taking the first two derivatives of position yields velocity and acceleration. In the case of a sine function, the second derivative is just the negative of the same function (see Figure 3).

POWERPNT_EPqmTS0lBD_9138d0d9bca07dd7cff0b53d7c182bbb22ec4a34.png

Figure 3. A sine function and its two derivatives.

From this, it becomes apparent that peaks in the linear acceleration of the drumstick correspond to valleys in its position, and therefore to drum strikes. We can take the acceleration measurements from the accelerometer and use a peak-finding algorithm to find the peaks and detect drum strikes. The simplest way to do this is to take the first time derivative of acceleration and look for zero-crossings. Unfortunately, this method would also detect noise and all kinds of other false positives. To keep this article short, I will skip the details and put a block diagram of the improved peak-detector that filters these false positives (see Figure 4). In short, the algorithm filters peaks which: 1) are low in amplitude, 2) are too wide, 3) are too close in time.

chrome_nHagCRn1tt_fe5f59db319526c04d3999fad125da9af1f2a8de.png

Figure 4. The peak detector used in Drumless to detect strikes.

Velocity Detection

As in real drums, in air-drumming, we would like to have control over the loudness of a sound. Luckily, the loudness of the sound in drumming is proportional to the velocity of the drumstick before the drum strike (S. Dahl, 2005), and this information is readily available from the gyroscope. The only thing that has to be done is to convert the value from radians/second to an integer in the range [0, 127], which will be understood by MIDI.

Drum Detection

So far we have learnt how to detect a strike and how to determine how hard that strike was. What's left is to determine which type of drum has been struck. Intuitively, one would use the motion data from the sensors to obtain the position of the drumstick and use that to determine which drum has been struck. Unfortunately, obtaining an accurate position from these types of sensors is extremely hard (if not impossible) since the double integration of the acceleration data (which is noisy) leads to errors (measured in meters!) in just a couple of seconds. Finding accurate orientation, however, is achievable through a process called sensor fusion. In short, the angular velocity from the gyroscope is integrated once to obtain orientation. The accelerometer (which also measures acceleration due to gravity) is then used to find which way is down, and the magnetometer to find which way is North. With these two reference orientations, the error in orientation from the integrated gyroscope data can be corrected. Although all this is achievable, getting good results from a custom implementation would be time-consuming, to say the least. However, InvenSense provides libraries for their sensors that can do sensor fusion.

One way to represent orientation is in the format of roll, pitch, and yaw. These values tell the orientation in degrees around the x, y, and z axes respectively. In Drumless, the conditions shown in Figure 5 were used to determine which drum has been struck.

POWERPNT_cktxGLE55S_bdf2b1dcade80dd67e7567a16b553b8dd0c84982.png

Figure 5. The conditions used to determine which drum has been struck based on the orientation of the drumstick.

Conclusion

And that's all. As I said, the concept is really simple but it took me more than two years of experience, learning and prototyping until I made the final product that you saw in the video. I hope that this articled taught you something, inspired you to go and make your own project, and hopefully motivated you to be persistent and follow your passion.

I had to skip many details in this article to keep it short and only gave a high-level overview of how the different pieces come together. I have left hyperlinks to some of the relevant topics throughout the article. Lastly, if you are interested in this project and want to know more details, here is a link to my dissertation.

I have a BEng in Electrical & Electronic Engineering and an MSc in Artificial Intelligence. I like to combine knowledge from both fields in my work. In short, I make computers walk, talk, hear, see and understand.
DesignSpark Electrical Logolinkedin