Let's Design and Build a (mostly) Digital Theremin!

"the PDP 10 uses signed shift distances." - Dewster

The PDP 10!   ;-) Now that goes back a Looong way!  - My first encounter with a digital computer was the PDP 8, then the PDP 11... I didnt even know there was a PDP 10!  - as to whether the PDP 11 used signed shift distances, I could not say... All I ever did with them was to present a pack of punched cards to the operator, and wait a couple of days for the result of the computation (which was often a rejection because I made some syntax error in my Fortran).

It was those expieriences which got me to abandon the idea of a career in programming - Programming was one of the modules for my degree - but it was optional.. Back in those days programming was for business or scientific non-real-time stuff mostly - one didnt even imagine having an embedded processor in a circuit... Amazing how quickly it all changed! A few years after completing my studies the whole picture changed, and affordable processors one could place on moderately low cost boards became a reality - and programming became an almost essential skill for EE's.

But back to your HIVE / FPGA - I am almost coplletely ignorant about implementing high level functions in a FPGA - I have trouble working out how to program the independent ALU functions in the PSoC 3 and 5, and these are already configured... I presume that the code must be embedded in the verilog? Is there any good reference  or even "FPGA for idiots" books you could recomend ? -

I can see that a fast processor like Hive implemented in a low cost FPGA would have great advantage for what I hope to do - In particular, exponental operations to facilitate functions like my "On Key Emphesis" (where the volume is modified by how close one is to a semitone - as in, the volume is at maximim [constrained by the volume antenna input] when one is on-key, and at minimum when between two semitones - with the gain being adjustable so one could even silence off-key notes - but the idea is to have a "middle ground" where one does not lose the ability to go off key, and where vibrato is only marginally affected, but there the volume feedback acts as a subliminal guide to the player to keep them in tune) and for implementing MIDI as seperate streams carrying pitch and volume data. My experiments with On-Key-Emphasis have been based on voltage quantizing using my voltage outputting front-end - Every 1/12th of a volt represents a semitone - and its quite easy to shove the voltage into a PSoC's ADC and get a 8 bit  MDAC output from this - but it would be nice to have a unit which accepted audio and performed the same function.

Fred.

"All I ever did with them was to present a pack of punched cards to the operator, and wait a couple of days for the result of the computation (which was often a rejection because I made some syntax error in my Fortran)." - FredM

I've heard an almost identical story from several others, and it kind of makes me glad that I dropped out of college early and missed that experience (went back later to find early Pentiums in the CS labs, thank god, though they had issues too).

"I presume that the code must be embedded in the verilog?"

Hive code is in a text file "boot_code.h" which specifies the initialization data for the main memory block RAM.  I use a lot of verilog constructs to make it human readable.  There is some example code at the end of the design document.

"Is there any good reference  or even "FPGA for idiots" books you could recomend ?"

Not that I've seen, which has made me often toy with the notion of writing one (my defunct web site had a "verilog corner" with several smallish verilog examples).  I have one of Ashenden's VHDL books and several others but can't recommend them.  The publishing field IMO is heavily contaminated by those who manage and teach managers, rather than those who do.  Digital design in FPGAs can be a lot of fun, but as in any field there are those who's primary function is that of killjoy.  They drone on and on about process flows, test benches, etc. until your eyes glaze over.  They should be spending the time showing you how to diagram digital structures on paper.  The Hive paper shows some of this diagramming, where vertical lines represent ff's / registers, with async logic in between.  Then you draw waveforms on paper to "simulate" what gets where and when, with the vertical lines in this case being rising clock edges.  Without these two on-paper skills one really can't design anything of substance, because one can't "think" about these things without mental crutches.  Or I can't anyway, even after doing it full-time for more than a decade.  The simplest thing will stymie me if I don't diagram.

This site is pretty neat: http://www.fpga4fun.com/index.html

The author has the right attitude and seems competent, but I find his coding style a bit confusing (then again I dislike most of the HDL coding styles that I encounter - I like to make clocked blocks look like separate blocks of code, with the more complex async logic located in between them).  He shows you examples in both verilog and VHDL, and you might see there why I prefer verilog (less verbose and more C-like).

" ... In particular, exponental operations to facilitate functions like my "On Key Emphesis ... and for implementing MIDI as seperate streams carrying pitch and volume data."

Aieee!  "On Key Emphasis" via volume is a brilliant idea!  My crummy brain never got past drones, pitch correction, and LED displays as on-note helpers.  Where do you get all these fantastic ideas - from a pact with the Theremin devil? ;-)

The volume emphasis function would obviously be easier if the current pitch was a number inside the FPGA.  And I haven't messed with the volume side at all yet, though I imagine log/exponential functions will come into play (as they do on the pitch side).  Inter-note, the volume control value could perhaps be a linear function of the pitch number, as the frequency difference between semitones is 2^(1/12)= ~1.06

Outputting MIDI from an FPGA is child's play, I have some really nice UARTs (been working on UARTs for what seems like ever) but generating the stream of MIDI data might be slow & complex enough that it would benefit from a processor construct.

Fred, I'd be more than happy to help you put an FPGA in your Theremin if you are thinking that may be a solution.  One of the big tricks is to employ timing to get data into and out of the FPGA rather than AD/DA (for pitch numbers, my DPLL accomplishes this via relative I/O phase).

"All I ever did with them was to present a pack of punched cards to the operator, and wait a couple of days for the result of the computation (which was often a rejection because I made some syntax error in my Fortran)." - FredM

Little O.T. story from my punch card and paper tape days... 

At college one of our projects was to learn to control the graph plotter connected to the Pr1me mainframe using Fortran 77 and a graphics subroutine library and demonstrate our understanding by drawing a picture.

I implemented a tiny subset of turtle graphics (HOME, PENUP, PENDOWN, FORWARD, RIGHT) and a stack (PUSH, POP) to overcome FORTRAN 77's lack of recursion in about half an hour, then spent the rest of the day interactively creating graphics procedures in Apple LOGO on an Apple IIe and transcoding them into FORTRAN 77 once they worked. (Row of houses with doors and windows drawn in perspective, hexagonal bee hive with stylised bees, dragon curve, Sierpiński carpet, Lévy C curve and a sort of fractal pentagram design.) Then I went to the pub.

A fortnight later, most of the other students had managed to draw a stick man.

I was marked down hard because my code looked nothing like "proper" FORTRAN coding.

"I was marked down hard because my code looked nothing like "proper" FORTRAN coding." - GordonC

Yeah - Sadly there is a lot of that! - IMO Those who actually think outside the box and innovate solutions for a given task get marked down by those who couldnt innovate to save their lives, and who are afraid to even look, let alone step outside their box... It doesnt just happen at uni - it happens everywhere, even here! :-(

"Where do you get all these .....from a pact with the Theremin devil? ;-)" - Dewster

LOL ;-) .... No, certainly not!  - At least one person from TW has seen me for who I truly am, and publicly exposed me on his web site.. I AM the theremin devil !!! (well, actually, an alien "overlord" as per A.C.Clarks "Childhoods end" :-)
>:-)

OFF Topic from here on!

The OKE scheme came to me the way a lot of ideas come to me - some discussion, often here on TW, gets me thinking - I think it was a combination of discussion about pitch preview combined with a separate discussion on Levnet about auto-tune which started me thinking - I deliberately cram my mind with any technical problems I have, every night before I go to sleep.. For particularly difficult or confusing problems where every solution I can think of has a big 'down side' I will go through the problems / solutions / downsides first, systematically - then deliberately 'discard' them and go over the actual goal I am seeking - I almost "ask" or "instruct" my brain to find anything I have not thought of, no matter how absurd.. This is an almost meditative exercise I have honed over years..

The answer isn't always there when I wake up, but almost every "big" idea I have comes from this process - I will wake up with something I never dreamed (LOL) of before, sometimes the next morning, sometimes a week later, often in the middle of my sleep I wake up with "the answer" and have a notepad and pen ready to write these down .. Sadly, quite often what I have written is completely incomprehensible! - Also, I dont always get any sane "answers".

I was really into neural networks years ago, and read about someone who invented a neural network computer and discovered by accident that when input to the NN was disconnected it went to a "dream like" state, producing "results" from the data it had previously 'examined' - These results were bizarre, but had qualities this inventor recognized as useful - He then added a NN to take the "bizarre" data, and configured this as a "sanity filter" - it could, from the huge quantity of mostly useless data, extract possibly useful data.

The inventor was a materials scientist, and feeding data on material properties, putting the nn into a "dream" state, and then tuning the sanity filter, he obtained results which he did not believe - but tested.. The result was a whole cache of new materials he could never have thought of.. and which he went on to patent...

I read the above  article in a hospital waiting room, on one of the rare times I forgot to bring something to read, and there was actually something there worth reading!  I was never able to find the article again, and do not even remember what journal it was in.... My memory of the content is probably flawed... But it profoundly affected me ... I thought - "I have a powerful NN in my head - If a (comparatively) simple NN can produce such results, I should be able to emulate this".. So I started playing with the idea -

The biggest problem is the "sanity filter" LOL ;-) ... Knowing what to reject - because if I chased every idea which came into my head I would have been certified many years ago!  ;-)

the OKE idea came about because of simultaneous discussions here and on Levnet relating to two quite different topics - one being preview, the other being auto-tuning.. I put both matters into my NN one evening, with no thought about a link between them .. The core idea came the next day in the context of increasing preview volume as one got more "on key" - this was not a good solution I thought, as one really needs to hear the preview tone as one hunts for the right pitch... But I soon realized that OKE might have far greater potential, and might be a simple viable and less "destructive" means of getting some form of "auto-tune" (or at least a less intrusive way of providing "on-key" reference points.

Even after having the idea, it took me quite a while to decide that it may be worth pursuing.. Its been "on the shelf" for quite a long time (I think I floated the idea on Levnet oh, probably 2 years ago) - but with going to a CV outputting front-end, implementing it becomes a lot easier.. I think that it might add a whole new spectrum of sounds to the theremin ranging from slight emphasis to "harp" type sounds where the note only sounds when the tuning is within perhaps +/- 5 cents.

I imagine OKE would be really easy to implement in a digital theremin, and that with a bit of processing to detect the "on key" duration could actually prevent transitory notes (as one sweeps rapidly to the wanted note) from sounding at all... If the player is monitoring both a conventional preview (I don't like the term preview - I think it should be called cue) and the OKE "main" output, I think that manually optimized "auto tune" could be incorporated quite easily, and if applied gently enough would not remove the sonic qualities which make the theremins sound so special.

"My crummy brain never got past ..."

I believe that our brains, even the least bright ones, are capable of far greater things than we ever allow them to achieve ... IMO, its not the hardware that's the problem, its the software! - We have an OS which has been programmed incrementally without guidance - there was no intelligent design involved, it simply developed to cater for the survival goal, (through the crudest of mechanisms - evolution / gene propogation) - which included all the insane social constructs that developed. "hardware" got "added" to meet the increasing complexity - but I think there is massive redundancy - huge computing blocks which are rarely used an which can only be accessed if we hack our way out of our OS and get down to 'ASM' level...

Alas, we dont know enough about the hardware (let alone the software) yet to truly utilize it - Our attempts (NLP etc) are only skimming the surface of what could be possible IMO - A bit like modifying our "autoexec.bat" but never writing any TSR routines or using (in my case) the math's co-processor. It seems to me though that there is a somewhat different "OS" operating at the subconscious level - that it can take a bunch of "punched cards" and often finds ways to make use of this data - I think its main problem is communicating any results to the foreground OS... "We" are (or think we are) mostly focussed in this "foreground" and IMO it tends to reject anything that doesnt fit its primary paradyms, even when this comes from the background "OS" or "inner voice".

I have also noticed that my ideas are of far lower quality when I am healthy - When I have had a major hypoglycaemic attack or some illness is impairing my thinking, I have the best ideas - I dont really understand this at all - Perhaps illness weakens the power of the "foreground" OS, or perhaps some other mechanism is at work.. Its a real pain that its this way 'round though, as the effort to re-capture some of these ideas when I get better can be great, and a lot of stuff gets lost.

 
Fred.

"I AM the theremin devil !"  - FredM

Ha ha!  (I see your devil avatar is back.)  You should at the very least make people sign their souls away before giving them all of your good ideas! 

I wrote a paper in college on neural nets which spurred a lifelong interest in them, though I remember some TV show from my youth where some engineer was talking words into a microphone, training a NN to recognize them (seemed like magic to me at the time, the commentator said he pressed either a "punishment" or "reward" button for each response which seemed cruel to me, but of course it was just back propagation at work).  Read quite a few papers getting up to speed (that's the main thing I miss about college: sitting around filling my brain with all of that IEEE stuff behind the paywall). 

Once you understand how NNs work a lot of brain activity seems clear, stuff like the white noise of the shower stimulating thought.  And it's maybe not so weird that brains scale so well, from the tiniest insect to us.

I have a similar creative activity to you Fred, but probably not quite so purposely honed.  I find if I just keep mentally juggling something over a period of time that technical issues seem to solve themselves in an unconscious fashion.  Works with crosswords too!  But the intense preoccupation it requires tends to cut into personal relationships.  The absent-minded professor generalization makes a lot of sense to me, it takes an internal "Manhattan Project" to move forward on anything complex.

Have a copy of Minsky's "Society of Mind" from the thrift store for $1 or something. A more general consumption book from a while back was "Brainmakers" which described several approaches being taken at the time, one of which was "Cyc" which I see is now part of some anti-terrorist (i.e. government spying on the innocent) program.  IIRC one of the programs had a "focus" lever of some sort that could go from generalizing to the specific.  Good book, but dumbed down to the point where it might actually confuse the more technical among us.

A book I read lately was "On Intelligence" which consists of a SW engineer describing the the brain.  It seems a bit rushed and repeats itself (you may come to hate the word "saccade") but I learned a lot from that one.

"Robot" is an incredible read (I've bought this book like 4 times after lending it out and not getting it back).  A book on robot AI by a free-wheeling robotics engineer.

After setup / breakdown for two piano recitals here at casa del dewster today I'm too beat to type.

"You should at the very least make people sign their souls away before giving them all of your good ideas! " - Dewster

LOL ;-) .... Nothing is worth the blood its signed in! there are entities with more power than the gods or devils, theyr called lawyers and solicitors and barristers and judges - or the catch-all.. "Suits".... Apart from which, im not sure that I believe in the "soul" - I think possibly there is the NN, the whole NN, and nothing but the NN !

"But the intense preoccupation it requires tends to cut into personal relationships.  The absent-minded professor generalization makes a lot of sense to me, it takes an internal "Manhattan Project" to move forward on anything complex." - Dewster

"tends to cut into personal relationships" - Yeah. :-(

Somewhere, mention was made of noise.. I mustnt get into this stuff again - its one of the subjects I deliberately abandoned because I become obsessive about it - almost lost myself in it back in the early 80's when working at the medical school.. but noise does seem to be a major factor in creativity - be this from audio input or optical or electronic stimulation, and I built many experimental bits of kit.. often with / for medical students.. and often these students concocted psychotropic substances to "assist" these experiments..

those were interesting days... ;-) .... But the "internal "Manhattan Project" takes some casualties - The way I "do it" now is less fun, but far safer.

Anyway, I am feeling quite pleased today! ... I just managed to sell one of my ideas for £10k and retain a % of any royalties from it ( I tried to negotiate a larger payment and less/no royalty, but failed to ):  - This was from an Idea I put almost no conscious effort into - I was presented with an absurd "impossible" request and almost jokingly gave it to my subconscious before I went to sleep - woke up with the seed of a completely crazy solution, spent the time simulating and perfecting it, filed a patent application (with absolutely no intention of going any further with it - I just file an application for almost anything I think may be "inventive" - its free to do this for a UK patent, and it gives me a year in which I can communicate the idea with some protection behind it), and presented the idea to the client -

Got a Skype call from Turkey this am and an email asking me to release the rights to my uk application (transfer it) so a WIPO patent can be filed, in exchange for £10k and a good percentage of royalties - they take it and all the hassle and expense of the patent process and defending it, I get up-front money regardless of whether the patent is granted or not, and a royalty if it ever makes any money...

And if the idea does go into production, there may be several "theremin" related circuits in the wheel of a future electric vehicle you buy!

That, to me, is concrete proof that its worth playing mind games!  ;-) .. this is only the 2nd time I have made any money from an idea though.. but particularly satisfying because I would never have had the idea if I hadnt explored theremin circuitry - so its some small return, possibly more than I will ever make on theremins! ;-)

Fred.

"That, to me, is concrete proof that its worth playing mind games!  ;-) .. this is only the 2nd time I have made any money from an idea though.. but particularly satisfying because I would never have had the idea if I hadnt explored theremin circuitry - so its some small return, possibly more than I will ever make on theremins! ;-) '  - FredM

Fred, congratulations on your Theremin-related £10k idea!  Make sure you reward that part of your NN with some constructive back propagation!

I'm not sure what it is about the Theremin that makes it such rich ground for general research and enlightenment (though magnetics tend to be largely overlooked by most modern EEs).  Was over at a friend's house this evening playing with the capacitive controls on their new kitchen stove for a few moments.  Something fascinating about touch sensitivity that causes everyone to think of it as the height of modernity.

"Make sure you reward that part of your NN with some constructive back propagation!" -

Yes - This is an important aspect of training a NN.. But I have never deliberately done this - simply dont know enough about myself / my NN , to know where to start.. However, I suspect that my NN doesnt enjoy stress - and that the relief from stress that getting money brings with it, is a reward mechanism... The "loop" is quite closed I think really - Good ideas (even if they dont bring in money) are pleasurable - and I think the part which "produced" these ideas gets reward from this - add a survival component (money) and I think the reward factor could be greater.... But I certainly get more of a "buzz" from coming up with an idea than I get from the financial reward..... And it seems that the more "buzz" I get, the more ideas "appear" - so I think this is the primary "constructive back propagation".

" Something fascinating about touch sensitivity that causes everyone to think of it as the height of modernity." - Dewster

LOL - That is true! .. And often IMO, touch controls are a real pain - they are getting better - but I still strongly prefer a good quality mechanical switch! - The fact is though that good mechanical switches are expensive and touch technology is becoming extremely cheap, particularly when a large quantity of product is manufactured.

I cannot detail much about my 10k idea, but it had nothing to do with any normal use of capacitive sensing - suffice it to say that capacitance can be used to "communicate" data between two completely sealed units which are in close proximity, and can do this even in the presence of extreme and chaotic magnetic fields - it can even do this when the sealed units are moving with respect to each other if there are always overlapping plates - using theremin techniques, the respective positions of the plates can be determined and this data used to compensate for environmental factors as well as doing other things..

The whole field of what can be achieved - industrial application - with a few strips of copper and some "theremin" electronics, hasnt even really begun IMO... Its not the obvious stuff people can see (touch switches, automated taps etc) which will be the next 'revolution' in capacitive sensing (impressive as these may be) IMO, it could be the use of hidden sensors in contained environments..

I have several other patent applications  being looked at by clients - ranging from joysticks to enclosed pressure sensors - and applications (mostly for safety critical applications where capacitive sensing is used simultaniously with some different sensing technology) -

 For example, I added capacitive sensing to a grip detector which had used force sensing resistors (FSR) - this is used on the control stick of a large demolition vehicle, so that if the stick is accidentally bumped, the wrecking ball doesnt go beserk... The client had a failure of the FSR in some machines, and I added capacitance sensing - both sensors are ANDed now, and the hand must be sensed for a predetermined time before activation can occur.. Simple and unseen..... I have implemented similar detectors for nuclear reprocessing plants, where the sensors output a data stream which is compared to ensure the data is valid, and the verified data is analysed (a process I have not been part of)  to to detect fatigue in the operator.

IMO, there are a huge range of applications only starting to appear, and which in many cases will one will not even be aware of - For example I know of one vehicle manufacturer who uses (or certainly was planning to use)  capacitive profiling to "map" occupancy in the vehicle -

I only pick up a tiny fraction of the capacitive sensing jobs people enquire about - mostly they require me to be at their company, and work in conditions that I am not able to for health reasons - I can only work from home with a few visits to the client at best - but awareness of capacitance sensing and what this offers is growing - in this respect I think that understanding this field and the expierience gained playing with theremins could be really valuable to those going into electronic engineering as a career. For simple short-range applications (touch switches and sliders etc) no special skills or understanding is needed - its within the scope of "normal" EE work, and cheap  off-the-shelf solutions are available..

But for the growing area where these off-the-shelf solutions do not fit the bill, particularly with longer range precision sensing or capacitive "imaging", theremin techniques (in particular use of inductance and tuned sensors - NOT heterodyning!) and understanding could be a valuable asset.

 Fred.

"And often IMO, touch controls are a real pain - they are getting better - but I still strongly prefer a good quality mechanical switch! - The fact is though that good mechanical switches are expensive and touch technology is becoming extremely cheap, particularly when a large quantity of product is manufactured."  - FredM

I agree.  The stove was fun to play with, all those blue LEDs glowing back at me, but a knob is easier and more intuitive to use.  They should at least put tactile grooves where the sliders exist; and the sensitivity could be higher, I had to press my finger into the glass some to get a response.  It makes for a nice sealed surface on a splatter-prone stove top though.  My personal favorite control for most things electronic is the rotary encoder, and I plan to have at least one on my prototype.

"They should at least put tactile grooves where the sliders exist; and the sensitivity could be higher, I had to press my finger into the glass some to get a response." - Dewster

This is the big problem - the balance between sensitivity and immunity from accidental triggering.. All the same problems one finds with theremins (effect of the environment etc) apply to these switches - except, of course, that the range is much reduced and only two states need to be determined.. Grooves are really discouraged simply because they can collect / trap unequal distributions of dirt or moisture - The advice is to keep the panels smooth and activation positions somewhat ambiguous, so one gets a more evenly distributed layer of muck on the surface! ;-)

<Nothing about theremins... Only boring tech stuff and hot air follows! >

If one has ever played with a development system for capacitive sensing (something like the PSoC Capsense for example) and observed the poor signal / noise one can get, and configured the automatic 'background' recalibration, one sees that its always something of a compromise ..  As I say, its getting better all the time - but there is no way I will ever put capacitive sensing on its own in a critical application.

IMO, there is a way to make absolutely reliable use of cap sensing, and get better than mechanical switch operation for lower cost than a mechanical switch.. its not so "cool" - but its tactile and reliable - and this is to have the capacitive plates embedded in a flexible material, ideally with a mechanical 'click' action.. this way the user pushes a sealed tactile surface - but it is not the capacitance from the finger that activates the sensor, its the capacitance of the moving plate within the flexible panel that does it - this plate can in fact be a continuous grounded flexible covering, with a shaped dielectric / insulating panel behind it (for example a sheet of rigid plastic with holes cut where  PCB sensor pads / actuator for tactile feedback are placed.. This scheme is perfect - in mass production it is cheap even for complex panels, and it shields the instrument from incoming and escaping EMI.

Alas - its not so "cool" - I designed the control system for a food processor using the above - ten full pre-production prototypes were built - it breezed through all the certifications - everyone (including me) who used it loved it.... But at the last moment before Chinese production started, a big boss squashed it because it wasnt his idea of "the modern image he wanted for his products". They went for a standard capsense implementation, then 2008 hit and they went bust. The thing which really piss*d me off is that I had to wait for more than a year to get money owed to me paid by the liquidators - and they wouldnt even let me have one of the prototypes - these were all gathered by the entity which acquired the IP!

When it comes to capacitive sensing, I think that by far the best products have been spin-offs (the Quantum Research company which produced the Q-Touch sensor IC and which is now owned by Atmel ) from the University of Southampton's research. They are now working on a sealed integrated capacitive pressure sensor which, if cheap enough, will I think change the way people view capacitive switches - because they wont even know they are using capacitive switches! ;-)

Fred.

 BTW - The QT IC's were good compared to what other cap sensors were available at the time - I have no idea how they compare with the competitors at this time.. I was briefly involved with QT on one application where a PSoC client was not able to get what they needed from CapSense, and I visited Quantum and moved my client over to the QT parts after having seen how (comparatively) good these were - but CapSense has advanced greatly since those days.