Let's Design and Build a (mostly) Digital Theremin!

More violin: [MP3]  Just played this for the 6th time or so, starts on the lowest violin note and is in key of C, so it's revealing & simple.  The tempo is somewhat off because I'm super sleepy - wish I was a better player.  

Trying to simulate a particular voice really makes you appreciate all the nuances of it, you have to really listen to understand what's going on.  I don't think I ever realized how rich and beautiful sounding the violin is.  It's a crazy moderately high Q multi-resonant box.

Even though it only has 2 axes, the Theremin is a very natural controller for a lot of synthesis - much better than a keyboard for many (most?) monophonic voices IMO.

Been playing with the filter code today, and adjusted this violin patch by watching the spectral display with noise going through the filter bank to even up the amplitudes and eliminate the gap.  Using a 2nd order HP filter rather than 4th for the non-harmonic resonator and a lower cutoff frequency.  I've heard it so much today that I'm probably going deaf to the violin sound, but a perfectly passable patch seems to only require a reasonable pile of somewhat disorderly resonances driven by a slightly mellow sawtooth.  Like human vocals, it's fairly low-hanging fruit for relatively simple synthesis hardware / software.  And also like human vocals, it's a blast to play.  Job #1 for any instrument is to have good tone, otherwise no one wants to listen, including the musician.

Even more violin!: [MP3]  (same opening interval as the previous, which is what brought it to mind...)

Worked on the SVF filters again this afternoon, turned the damping knobs into resonance knobs: (1-damping)^2, where resonance = 0 gives a damping of 2 (less than critical damping) and resonance = 63 gives more or less infinite ringing.  I've wanted to do this for a long time, as "turning it up" should give more "effect" and make it ring longer, and not shorter as a damping knob does (following the Python principle of least astonishment).  Stuck with damping this long because, to me anyway, it's physically "realer" than Q in terms of what's going on inside the filter, and the limits are more constrained [0:2] and therefore tractable to implement.  I'm not labeling the knobs 'Q' because the number displayed technically isn't that (technically Q = 1/damping).  But who cares.

I know I'm no virtuoso, and the significantly lowered sensitivity and pitch correction do help, but I can't imagine playing anywhere near this well without the pitch display. If it's a crutch, it's one I believe I can live with long term.  Anything to make the Theremin less of a harsh mistress.

Great! Violin sounds more and more realistic.

"Great! Violin sounds more and more realistic."  - Buggins

Thanks!  Vibrato and slides in particular reveal the many resonances.

===========

More About All-Pass Filters

Back to the diagrams:

As I stated before, all of these forms have the same transfer function: y/x = [(z^-n) - g] / [1 - g*(z^-n)].

If you mark the input to the delay as 'A' and solve for the transfer function of A/x you get:

1:  A/x = 1 / [1 - g*(z^-n)]
2:  A/x = [1 - g^2] / [1 - g*(z^-n)]
3:  A/x = 1 / [1 - g*(z^-n)]

So 1 & 3 have the same amplitude and phase response at A.  2 shares the same denominator with 1 & 3, but the numerator has a gain factor of (1- g^2), which should lower the amplitudes of the delay line samples.  Indeed, if you graph and compare the frequency responses, the delay line peak amplitudes for 2 are bounded by +6dB (gain of 2) as g approaches 1, whereas the peak amplitudes for 1 & 3 are unbounded as g approaches 1.  This seems to be a good reason to prefer 2 as the filter topology to implement, as the storage upper dynamic range is constrained.

It's not at all unusual for digital filter designers to favor certain topologies over others for these types of reasons.  Internal overload happens in the analog world too, but it doesn't seem to get discussed as much.  Though overload in digital systems can be more catastrophic (modulo rather than limiting).

[EDIT] Just noticed with topology 1: if you move the (1 - g^2) to before the delay & feedback you get the A/x transfer function of 2.

===========

More On Violin Synthesis

Now that I've had some time to think about the research papers I've read, and experimented a bit with what they're doing, I believe I'm beginning to get a feel for the underlying and unifying concepts at work.

One violin synthesis paper uses explicit bandpass filters for the lowest formants and a waveguide mesh for the remaining formants above that.  The lower violin formants are likely the primary resonant modes of the body but, since the body is constructed of 3D objects, the harmonics of these resonances are non-harmonic (i.e. not n, 2n, 3n, etc.).

The reverb papers often use a bank of comb filters in parallel (because you don't want the resonances of one to be crushed by the response of the others) feeding several allpass filters in series with each other, or nested within each other, or both.  Now, each comb gives you a series of resonances, evenly spaced by Fs / n (sampling frequency / comb delay) but the even spacing is problematic because the ear easily identifies that sort of thing, so the allpass filters are there to knock the resonances off of their even spacing (but this is tricky to tune in order to not sound comb-like because the allpass filters are also acting in a systematic fashion).  Since they also have significant delay elements and are recursive (have feedback), the allpass filters also increase the density of the echos circulating through it all - but the allpass filters don't increase the number of resonances (this is key).

For violin synthesis we are looking for rather the opposite of this.  For each fundamental resonance we want the harmonics count to be roughly that of the harmonic series, and for the harmonics to be somewhat randomly located in the harmonic space above the fundamental.  And we don't want to hear any recirculating echos.  So there are at least two ways to approach the use of explicit formants coupled to "fake" harmonics.  The first is in the paper I mentioned above, have a mesh (~many interconnected combs and allpass filters) where the element delays are short enough for the resonance of the whole to begin where the formant resonances end (though they use a highpass filter to do the blending).  The second is to use a lower frequency comb so that the resonance spacing is dense enough, with one or more internal nested allpass filters to jumble them up, and cut the thing off with a high pass filter somewhere around the highest fundamental formant so as not to compete on the low end with the formants (this is what my prototype is doing now).  A potential problem with the second approach is the ear flagging the long delays, though the formant filters themselves often sound eerily delay-ish to me.

Maybe somehow have formant filters with non-harmonic partials, and just add a handful of these together?

p.s. I've seen other reverb topologies that are essentially the non-harmonic resonator writ small:  many allpass filters in series with each other and nested within each other, with a series delay and global feedback - which forms an outer comb.

All For Naught (0)

So I'm fooling around with the inharmonic resonator, trying to figure out the practical meaning of zeros (dips) in the frequency response.  If I give it some feed forward gain it produces comb filter zeros, which enhance the dips between the poles (peaks).  The audible effect is to make the resonant peaks steeper, in other words it increases the effective Q.  If I remove the feed forward gain (remove the zeros) and then dial up the feedback gain (increase the pole amplitudes = +Q) to compensate, the result sounds the same to me to the former lower Q + zeros.

Vocal and violin synthesis / characterization papers sometimes mention zeros but don't really deal with them meaningfully.  One way or another they figure out what the poles are, but then either assume there are no zeros in the vocal tract response, or throw their hands up in the air with the violin zeros.  It's kind of strange, poles and zeros show up almost equally in electrical and digital filter theory, which are largely analogs of physical things, so why the lack of zeros and/or interest in them when analyzing actual structures?

In Excel, the resonator response without zeros looks a lot like a bunch of band pass filters, where the skirts level off between resonances.  With zeros the transitions between resonances dive much deeper, and I'm wondering if that is audible / important / critical / unrealistic to a physical resonator like the violin body?  I'm kind of thinking zeros, outside of the internal filter operation, maybe aren't all that important.  If you even casually mess with the various types of filters and orders of them you'll find that narrow, high Q poles are quite easy to hear, but notches are more or less inaudible unless they're fairly wide / low Q.

As I pointed out earlier, allpass filters / reverberators can be designed to naturally have relatively low peak internal signal levels, even when the Q is high, though the same isn't true for comb filters. William Grant Gardner's Master's Thesis "The Virtual Acoustic Room" from 1982 discusses taking the output of a reverberator from the internal point just before the delay, and rejects this due to the "lumpy" non-allpass output it gives, but that's pretty much what we're looking for in an inharmonic resonator.

Anyway, the approach I'm currently looking into for inharmonic resonance is two or three nested allpass resonators + delays, with internal node output and no combs anywhere.  I'll also try sticking a low pass filter in the outer feedback path to make the higher frequencies decay faster, which they do for reverbs and other resonant physical models based on comb filtering.

Ideally, the gain vs. Q for the inharmonic resonator would be the same as that for the formant SVF filters, so making global adjustments to Q would keep the normalized levels somewhat even between all the filters and resonator.  So that's a design goal too.

Resonator

After some experimentation, I've concluded the outer resonator should be a comb with no zeros (no feedforward path), and the inner resonator unchanged as an allpass.  The lack of feedforward helps the inharmonic resonator blend with other filters in parallel (otherwise their Q gets diluted).  Here's what I've currently got going on the prototype:

Basically the nonharmonic resonator from that paper, but the inner allpass resonator is the type that constrains inner amplitudes to < +6dB, and I've added a feedback lowpass filter.  The Kn and Km knob parameters are signed (-0.5:+0.5) so I've incorporated their limited ranges into internal scaling that operates the inner resonator at 1/2 amplitude.  The Kn parameter is used linearly.  The Km resonance parameter is a flipped, squared, flipped type which gives expanded range at the maxima.  The lowpass w cutoff parameter is unsigned and currently just squared to better fit an exponential frequency type curve, but I may make it polynomial corrected and truly exponential.

Note the output is taken before the delays, and this helps to minimize phase issues when mixing it with the conventional SVF based formants.  Also note the signed saturation points, which keep it from going ape shit (modulo) with input/internal overload (really glad I added the tags and logic to do single cycle limiting / saturation in the processor stacks & ALU).  The added first order lowpass filter in the feedback really tames high frequency resonances from unrealistically running on as long or longer than the lower frequency resonances.  With it, you almost don't need Km, as dialing down the cutoff frequency damps things considerably.

Not shown here is the second or 4th order highpass filter feeding it, which keeps it away from the SVF formants.  Also not shown is some form of in/out gain to match the other formant levels (though it tracks the SVF formants pretty well with Q) and maybe some series / parallel routing to use it instead as a cheesy delay type thing instead of a resonator. 

I still haven't tried nesting two allpass reverberators internally, will do that soon.  Wanted to get the basics here up and running and in good shape, and get some experience with it all before I do.

Here's a 6 formant cello patch with the resonator in parallel (with external reverb): [MP3].  I'm learning more about the natural ranges of these instruments so as not to go below any pitch they can't physically do, which helps with realism.  Here's the same number played 1/2 step above the lowest cello note (C2, no reverb added): [MP3].

[EDIT] The diagram has two small errors:
1. The arrow head going into the right side of the lowest z^-1 box should instead be going into the left side.
2. The leftmost summing node input should have an arrow head on the lower (feedback) input.

Hi Dewster,

The cello mp3 in your last post above has me returning to press you into making your system, as it stands now, available for buying, borrowing, or stealing. 

I don't know if we talked about where you are ultimately going with this, but it sounded like you were receptive to putting some preliminary systems out there, either supplied by you or as DIY projects.  I would be willing to go either way - buy something from you or build it myself from your documentation (as long as I know where that is in the past 1586 posts), and I'm the easiest customer of all - I don't need an enclosure of any sort.  I'm flexible regarding whatever you are willing to do, but if it's going to happen it would need to be sooner rather than later. 

Let me know what you think here or via email if you prefer.

Roger H

PS: did I say I love the cello mp3 above?  I'm dangerously close to building a Haible String Filter (because: zero latency and impressive boards), but your cello might save me the effort.  Consider it... 

Hi Roger,

Which MP3 did you like better, the higher pitched one or the lower?  I'm not that great, but I think much of the impact is the performance and the piece. 

I am 100% receptive to preliminary models getting out there, and particularly to you as I'm certain your feedback would be invaluable.  It's a time budgeting quandary for me though.  I've never laid out a PWB so that might take me a month or so to pick a program, come up to speed on it, get comfortable with it all, and order boards.  But in that month I might be able to get most of the DSP software behind me for at least the first pass, and I'm really burning for that to happen.  Not saying what I will, won't, or can't do, just wondering what to do when :-)!  I could also spend a month or so just hand wiring two more prototypes...  I do need something more portable and robust to take out / send out for show and tell, but I haven't managed to burn up the original yet (knock wood).

I really wish you were closer, to make visiting easier.  Do you ever visit the NJ / NY area for other reasons, or might you want to for this project?  I think you would be happy with it, but I don't know, I'm so close to it all.  Building one would certainly be easier than an analog board with 40 variable Q band pass filters!  And if you really need 40 resonators I'm pretty sure that could be done digitally if they aren't articulated or modulated.  The SVF subroutine takes 14 cycles, and there are 468 cycles per thread per sample interval, so maybe 2 threads worth of real-time if it were coded efficiently.

Torturing (as tinkeringdude put it) the non-harmonic resonator is kinda fun: [MP3].  All the clipping / overloading going on there is probably in the filter, there was no clipping at my PC soundcard input.  I worked on the knob gains and delay scaling today, I think I'd like to give it a common pool of sample memory with taps so as to more efficiently utilize it.  I made clockwise movement of the delay knob decrease the delay / increase the filter frequency - it's more of a filter than a delay and I was always turning the delay knob the wrong way.

Eric

I like both of the samples, although I thought they were the same patch just played at different pitches.  The woody resonant sounds are very similar to what I have been hearing with the Haible analog string filter, and with a little additional processing there are some great possibilities.  Not to mention tweaks that you may be able to make in the patches themselves.

Never get to NJ, and I only travel by car or RV these days.  I'd love a hands-on, but I really need the traditional volume-sensing and possibly traditional rod and loop elements (don't be offended-I'd try plates, but there are many personal preferences to consider both in playability and aesthetics).  Ideally I would just like to build one and update it as you make changes.

Before I go much further I'll contact you via email first, but these latest cello sounds are making me want to drop everything (except my other little theremin project) and get started on this over the winter.  I understand that you are not in a position where you want to be stopping what you are doing to build hardware, and I can do that myself anyway.  But to even consider that I need to understand exactly what the hardware looks like. Depending on the complexity I would probably want to do some board layouts for home fabrication.  These may be of use to you for a backup prototype but they would not be suitable for obtaining production boards (too many alterations/concessions made for limitations of home processing).

Anyway, look for an email from me, probably tomorrow, and hopefully we can go from there.  This is really an outstanding body of work, and its time is coming.

Roger 

Hi Roger,

I believe both cello MP3s used the same preset settings.  With my monkeying around with the non-harmonic resonator code I've lost the exact settings for that part of the preset, but it isn't too difficult to dial something that sounds extremely similar back in.  The loss of presets during development is rather unfortunate, as one often stumbles upon crazy sounding configurations during the testing.

I play my Theremin with reversed volume sense, which seems more intuitive to me, but others have trained for years with the traditional volume sensing or desire it for other reasons, so this is a switchable option on the SYSTEM page of the menu (as is 50Hz mains hum squashing).  Traditional sense has the advantage that it is likely easier to perform the "quick cut & slow swell" operatic note change as the "quick cut" move is towards the antenna.  But it seems backwards to me, and with the sense reversed you don't have to manually shut the thing up when you move away from it.  I need to tweak the scaling of one of the axis parameters to give it 2x or 4x more resolution, which I believe is just a trivial int => float exponent adjustment.  As it is right now the axis gains and offsets need a bit of tweaking after the sensing is reversed, and I'd like to eliminate that as much as possible.  This was the main reason I added the final axis parameter a while back.  I've been putting off any non-critical path edits lately, but do track them in a README.TXT file in the code directory.

I promised you and TW a while back that I'd make better schematics and take some better close-up pix of the prototype details.  I can probably do that in a day if hand-drawn is OK.  

1. The oscillator boards are relatively simple, though there is a fair amount of interconnect over CAT5.  You might want to get some CAT5 board or chassis connectors to mount on the oscillator boards to make everything neater and more modular.  The oscillators have their own regulators, but everything else gets regulated 3.3V power from the FPGA board or uses ~unregulated USB 5V.  So power cycling shouldn't be an issue.  The coils are physically mounted to the plate antennas, which minimizes any vibrational C interaction between them.  The high impedance end of the coil is positioned nearest to the plate (though the end winding is an inch or so away as the plate "looks" like a big shorted winding to it), the other end of the coil is low impedance.  The CAT5 is a problem if you use the connectors on it as they are larger than the diameter of the cable - I was thinking of running the wire to the board through the case split point with a split rubber grommet or something for sealing and strain relief, which would put the cable connection inside the sealed box rather than at the box wall.

2. The encoder boards are just wires and you could easily wire them in the air if you prefer.  Or make subset boards with 2 or 4 encoders per board.  I recommend the arrangement I have as the knob layout corresponds physically with the display entries, and there is enough finger room between the encoders.  And obviously you want the knobs below the displays so as not to block them when adjusting them.

3. The LCD is just wires to the FPGA board, though there is a contrast pot you'll have to deal with somehow.  I soldered a tiny pot to the display board and drilled a hole for access (it's a prototype so it's supposed to be hideous!).  This is a purely 3.3V part, they make 5.5V and 3.3V/5V mixed boards, so beware.  The one I ordered from eBay was advertised as 3.3V, but arrived mixed, so they had to send me the components to convert it.

4. The LED "tuner" board is the most complex.  It has 13 white LEDs, 4 blue LEDs, all 10 mm high efficiency types, and a red high efficiency single diode type 7 segment display.  There are 3 serial to parallel LED driver ICs, 3 current setting resistors, and some decoupling, but it's really mostly just wires and the layout is entirely non-critical.  The LED "tuner" board should be mounted at the very top of the control box display area, as it gives critical performance feedback.  Watch what 7-segment display you purchase, you want the single diode type so the forward voltage isn't too high for the ICs to drive off of 5V.

5. I have a small "catch-all" board that has the EEPROM 8 pin dip, the SPDIF TX LED, and some power distribution.

I'll add another post(s) with more information and a parts list as it develops.  If you want to get started you should order the FPGA board, and FPGA programmer, the encoders, the LEDs, the LCD, the EEPROM, and the other ICs.  I can provide you with coils and the SPDIF TX LED if you like, though the coil values might have to change if you go with traditional antennas.

I would urge you to at least give plates a trial period as they are more sensitive (very good engineering reason).  And you might find them easier to see and play, though of course aesthetically they are different than the expectation.  Build it cheesy with plates, get it working and become familiar with it, then adapt it / cannibalize it for the final target enclosure(s).

There are prettier ways to make plate antennas.  You could use a thin rigid non-C substrate (not wood I wouldn't think, but you could certainly experiment around to see if wood is viable here) with aluminum tape or plate sandwiched inside, and the coil exposed or otherwise viewable sticking out the back (I think the coils are pretty and make it look like real engineering is going on) with a small box on the back holding the driver board.  The plate could be round, square, rectangular, oval, etc. without a lot of variation in the linearity of response.  My volume plate is a 3D 'L' shape and it seems to work fine.  You could use 3 layer PWB material with no external copper for the antenna - you could even put the driver electronics on the back if you removed the antenna plate copper from a small region.  Or use 1 or 2 layer and cover the front with a piece of plastic or stick the whole thing in a plastic box.

(Test for non-C material with a regular Theremin: wave a longish piece of it near the pitch antenna &  touch the antenna with it.)

Eric