A few years ago, we mused about whether augmented reality is the future of theremin playing. Fast forward a few years, and music duo Veserium is doing just that. Group founders Ray Li and Michael Ndubuisi are pushing the limits of gestural controlled music with a new instrument they call SoundSpace: a combination of hardware, software, and VR/AR technology that gives them unprecedented control during a performance.
Eager to learn more, I sat down with Ray to learn more about the project.
---
TW: How did the idea for Veserium come about?
RL: I'll start at the beginning. In college, I was an engineering major and a music minor. A combination of those interests led me to start making new musical instruments. The first I created was called The Sabre, which was inspired by the cello (my favorite instrument).
Instead of a bow, The Sabre featured a joystick that allowed me to control the sound quality. As I thought about how to improve The Sabre, I wondered if we could replace the joystick with a camera that could sense motions and gestures of your hand. Thinking more along this path led me to ask- why not get rid of the entire instrument?
TW: How does SoundSpace improve the gestural music experience? For example, what types of gestures can it recognize and how can those be applied to musical performance?
RL: From the beginning, I've been obsessed with shaping sound. The first version of what we have today was called "The Aura" and the idea was- what if we could hold sound like a tangible object in thin air? We could spread our hands apart to make the sound bigger (louder), we could close our fingers into a fist to muffle the sound, and we could twist it to distort the sound.
The computer nowadays can generate pretty much any sound imaginable. But we're still using outdated instruments to control those sounds. Most modern controllers are still based on traditional acoustic instruments. People forget that those instruments were designed with a very specific purpose- to physically make sound. Now that the mechanism to generating sound has been digitized, in my mind, the only thing that makes sense is to design an interface solely around maximizing the amount of control we have over the sound.
We're in a new era of electronic music, where producers are literally designing instruments (sounds) that people have never heard before. We can shape sounds to a crazy degree of detail, but there's no way to control that sound other than drawing out curves with a mouse and keyboard for how you want the sound to change over time.
For example, in a typical dubstep bass line, or some other sound, there can be literally 5-10 parameters or more changing in the sound at the same time. There's no way you can play that live.
But we can. The gloves are extremely accurate - they know our motions down to a millimeter. Our rotation as well. We have sensors on every finger. By using all of this information, and wrapping it into an interface we created ourselves, we are able to play these complex sounds live.
For me, it's all about expressive control of the timbre. I want to be able to sing with my hands. I want to play as expressively as a violinist controls the tension of the bow against the string... and I want to make it even more expressive, controlling not just the volume and the pitch, but also the intensity and the equalization and the waveshape of the sound.
There is nobody else in the world that I know of that has as much control over sound in real time as we do.
TW: Is SoundSpace specifically tailored towards electronic dance music, or is it applicable to other genres?
RL: We're focusing on making EDM music at the moment, though my background and greatest love is actually orchestral music. Mike loves hip hop and rock. We're still defining our musical style, but it will probably fall somewhere between pop EDM and dubstep.
Theoretically the technology can control any sound generated by the computer.
TW: What can you tell us about the technology behind SoundSpace?
RL: It's powered by Virtual Reality technology. We're using the HTC Vive as our backbone. We made the gloves ourselves, and the software is all our own as well.
TW: SoundSpace bares some similarity in concept to Imogen Heap's mi.mu glove project. How is SoundSpace different?
RL: Imogen's gloves are awesome, and we have admiration for her work as a musician and innovator. Our gloves are different in several ways, but the most notable is the depth of control which I mentioned above. Think of our instrument like a "super theremin." It's totally customizable. Imagine a theremin which has instead of 2 antennas, a bunch of different antennas, allowing you to control a lot of different things all at once.
Imogen's gloves were built to allow her to be more visually expressive on stage, and to be able to switch between instruments without needing a full set up on stage. It's awesome.
Our gloves are built to allow us to craft sound with more depth of control. Imogen's interface is more gestural, and ours are more about finesse and articulation of movement.
TW: Your website mentions you are both former engineers. What's your background and how has that helped with the project?
RL: Mike studied Computer Science at Cornell University and then went to work at Amazon for three years. I studied Applied Physics with a minor in music, and worked in the startup scene in Ohio for a few years.
Our technology backgrounds are what enabled us to bring to life our artistic vision. We think of tech as a creative tool, just like a painter uses paintbrushes for their art. Technology is part of our art. It is allow us to create the style of music that we want to be able to create.
TW: Do you have any plans to make the equipment available commercially?
RL: At the moment, we aren't selling the gloves or the technology. Perhaps we will in the future, but for now, we're focused on being the pioneers of a new type of expressive electronic music.
TW: What's next for Veserium?
RL: Content, content, content. We will be making a new video every week, and releasing new music as well. We'll really be working to establish our sound as artists and to grow our fanbase and give them more to enjoy! We're also planning an audition for America's Got Talent.
---
That's a wrap! As a final treat, check out this video of Ray covering Pink's "What About Us" using SoundSpace. The amount of expressive control he has over each note is incredible!
You can learn more about SoundSpace and Veserium at their website (veserium.com) or by checking out their YouTube channel.
