ETrombone
What is it?
The ETrombone is a digital musical instrument I created which attempts to mimic many of the aspects of playing a traditional trombone but which does not use a physical slide. It can be thought of as the electric guitar of trombones, though possibly with even greater flexibility in some aspects. This is a project that I hope to continue working on going forward.
How does it work?
Overview
A traditional trombone can be thought of as comprised of three main parts: the mouthpiece, the slide, and the bell. Air flows through a mouthpiece, through the slide, and then out the bell.
The ETrombone captures all of these components, but in a very different way. It uses the same mouthpiece as a normal trombone, but the mouthpiece is attached directly to a 'windpiece', which does not have a bell but rather a pickup microphone. The slide is completely virtual and is accomplished through hand tracking. A computer then uses the information from the "slide" and the microphone to synthesize the final sound.
The Slide
In this first prototype, the slide is implemented using a Valve Index controller and a pair of lighthouse base stations.
Using OpenXR via pyopenxr, the position of the controller can be obtained in real time. That position in 3-space is projected onto a virtual slide and returned as a float value representing how far out the user has moved the slide. The user calibrates this slide by moving to where they want first position to be located, pressing the A button, moving to sixth position, and releasing the A button.
It should be noted that OpenXR support does not exist for MacOS, only Windows and Linux. I used SteamVR as my OpenXR runtime, with the null driver activated to allow the use of the OpenXR headless extension so that there is no need to plug in a VR headset. Information on how to do that is found here.
The Windpiece
The windpiece is a length of half inch PVC tubing which I tuned to roughly a concert Bb by cutting it to size. In actuality it is two pieces of tubing held together at the bend by parts that I designed in CadQuery and printed on an Ender 5 Pro.
At one end there is a 3D-printed adapter which goes from the inner diameter of the tubing to the standard mouthpiece shank size. At the other end of the windpiece there is a condenser microphone wired to a Raspberry PI Pico W.
When the mouthpiece is buzzed, the air flows through the length of tube and resonates at one of the overtones for that length of tubing. This is exactly what happens with a normal trombone, which is entirely the point. Because of this long length of tubing, it feels very similar to playing an actual trombone in first position.
Microcontroller Code
The Pico microcontroller samples the microphone at 8192hz and uses a short-term Fourier transform to convert frames of input audio into pitch bins. The lowest and highest bins are filtered out to account for the poor quality of the microphone, but fortunately it functions well in the range of typical trombone notes.
If the loudness of the loudest frequency bin exceeds a minimum threshold, then the frequency corresponding to the average of the bin is transmitted over Bluetooth to my laptop.
All of this code is written in MicroPython, however since at the time of writing MicroPython does not have support for Bluetooth on the Pico W in the main branch, nor does it have STFT built in, I had to compile a custom version of the beta branch for Bluetooth support which also included ulab for the STFT computation and numerical efficiency.
Laptop Code
Other than running pyopenxr and receiving the Bluetooth communications from the microcontroller, the laptop is also responsible for generating the sound of the instrument. My original plan was to use a relatively new library from Spotify called Pedalboard as this library allows for the use of VST3 plugins in real time and with a very convenient API.
Unfortunately I later discovered that Pedalboard does not currently have support for VST instruments or MIDI. I explored other options such as DawDreamer, however this library does not have real time processing support as far as I can tell, and very little documentation to speak of.
After probably too many hours, I managed to add VST instrument and MIDI support to Pedalboard by patching it in at the JUCE API level (in C++). The hardest part was fixing the real time processing as Pedalboard's existing implementation ran afoul of JUCE's best practice for VST plugins. Since this was a multithreaded program, it took a very long time to debug. I have opened a pull request implementing these features but for the time being they can be found here.
I use the BBC Symphony Orchestra Discover Plugin as my 'synthesizer' of choice (it's more of a sample library), but in theory any well-implemented VST3 Synthesizer with MIDI support is compatible.
In the current implementation, the pitch information is used to determine the partial, and the slide position is used to determine the note within that partial. This is a very coarse way to do note selection given that a trombone is the most fine-grained instrument, but I leave that for future work.
You can view all of the project code here
Additional Features
I have additionally implemented a way to play with multiple voices at once, allowing for one musician to play with the sound of an ensemble.
I also have added another experimental feature where rotating your hand changes the volume of the instrument like turning a volume knob, however this feature requires more development work.
Demonstration
Future Work
In the future I hope to make many improvements and iterations on this design. I shall list the features I desire to add here:
- Add more effects relating to the orientation and/or positioning of the slide hand in 3-space.
- Adjust the pitch within a partial according to the actual recorded pitch
- Shrink the windpiece to be more portable and less obnoxious to hold
- Add a rechargeable battery
- Replace VR controller with a camera on the windpiece and AI hand tracking
- Build a better enclosure for the microcontroller
- Better mounting for the microphone
- Isolate the microphone and the end of the tube to prevent feedback from interfering with live performances
- Mute the end of the windpiece so that the less-than-desirable sound of a pipe with no bell cannot be heard
- Make efficiency improvements to all of the code
- Try out more synthesizer plugins
- Open source the 3D prints