Let's design and build cool (but expensive) FPGA based theremin

Received second LCD touch screen from Waveshare.
It has the same corrupted image as previous one.
Shows only 48 columns normally(although they look like shifted down by 1-2 pixels), the rest of screen is filled with pixels from first line.
$33+$33 spent for garbage.

UPD: looked once again at display demo code, found that DE polarity is Active Low. I didn't yet try this option.

Found bigger plastic box for design - space inside is enough, coils are far from other boards, antenna inputs are closer to antennas, far from PCBs since box is wider than previous one.
Built new plastic pipe construction with different mounting sizes, and got following:

OMG, in real life it looks even uglier!

Thinking about switching to my initial plywood laser cut cabinet design.

Finally I've managed to get my Waveshare 4.3inch capacitive touch LCD working.
Realized that HSYNC, VSYNC, DE should be active high, and HSYNC/HBP/HFP should be big.

LCD controller is working. 800x480x12bit color (+8 bits for future hardware drawing support for tuner position autodisplay).

I wrote small graphics library - simple drawing and text (TBD).
Test image on LCD:

The same code working on Simulator:

Simulator is cross-platfor Qt application.

Simulates LCD screen, encoders, button, pedals.
Theremin sensor is simulated using "mouse synth pad" - area at bottom - horizontal movement is pitch antenna frequency change, vertical - volume antenna frequency change.
Mouse position is de-linearized to simulate values from real theremin sensor.

Simulator supports audio simulation as well.
When new audio frame of length N is needed, it interpolates sensor signals from previous to new value for each sample. For each sample, puts interpolated pitch and volume sensor values to allow audio IRQ code read these values, calls audio IRQ, then retrieves value written to Line Out register by synthesizer audio IRQ.

Inside IRQ, values are read from sensors, scaled, linearized, transformed to note / volume.

Amplitude and frequency vibrato is supported.

Simple additive synthesizer is implemented. Waveform is represented as 256 harmonics (amplitude + phase). On each harmonic, filter is being applied -  (naturally, based on table of desired spectrum).
Written helpers to initialize settings for standard waveforms - triangle, square, sawtooth.
Tried to set filter to voice formants - don't hear the difference... Something is wrong with my filter.

Simulator allows to experiment with synthesis algorithms. Code debugged on simulator is the same as will be compiled to run on second core of Zynq.

Project is available on github. Anyone may try it (use QtCreator to build and run).

Playing with mouse is a bit hard. Wacom tablet instead of mouse is better.

I'm planning to add native support of wacom tablet, to allow changing of note by pen movement, and volume - by pen pressure instead of up/down movement.
Another nice option - get sensor data directly from FPGA board via USB serial and use them.

"Finally I've managed to get my Waveshare 4.3inch capacitive touch LCD working.

Realized that HSYNC, VSYNC, DE should be active high, and HSYNC/HBP/HFP should be big."  - buggins

I was thinking two bum displays in a row seemed rather unlikely.  Much more likely is bum documentation of the display interface!  Glad you got it working!

Sim as an app is a really interesting move.  Should allow you to develop the DSP synth stuff quite quickly, as well as presets.

Are your 256 harmonics freely assignable for frequency, or are they strict multiples of the fundamental?  If freely assignable, you could do bells, steel guitar strings, and other dispersive / non-harmonic sounds really easily.

You're putting a lot of work into this, and I really appreciate that you're sharing so much of it!

Sim as an app is a really interesting move.  Should allow you to develop the DSP synth stuff quite quickly, as well as presets.

100% of DSP code is now shared between emulator and Zynq ARM core app.
Emulator just calls common Audio IRQ handling code. (Now there is no synthesis parts inside DSP, but one of two 667MHz ARM cores is reserved for audio synth only. Zynq PS contains 192KB of dual port SRAM accessible from both cores, and synth application (core2 app) may be placed to SRAM to get rid of unpredictable DDR delays (SDRAM on Cora Z7 board is slow - uses 16bit interface).
Core 1 (GUI) controls synthesis using structure in shared memory.

If some FPGA based synth modules were added to design, emulator can be modified to simulate these hardware.

Having shared GUI code is very convenient, too. PC may be used to develop / debug all core0 functionality, including instrument preset editors and see how changing of settings affect synthesis w/o recompilation / reflashing of real FPGA board.

Current state of project: development of GUI library / C++.

BTW, if there is a nice possibility to debug code on device directly from Xilinx SDK IDE (eclipse + gdb).
To debug FPGA side functionality (when strange problems appear), I'm usually routing some signals to registers which can be read by PS (CPU). 

Are your 256 harmonics freely assignable for frequency, or are they strict multiples of the fundamental?  If freely assignable, you could do bells, steel guitar strings, and other dispersive / non-harmonic sounds really easily.

Now they are multiplies of f0 (phases of harmonics are just phase_f0 * K mod 2^32)
To allow independent frequencies, it's just necessary to add separate phase vars[256] and calculate frequencies/phase increments separately.

Everything is easy if fits into 667MHz/48KHz single audio sample budget. It's about 13000 of ARM5 clock cycles.
Otherwise, parts of processing has to be moved into FPGA (10-15% of FPGA resources are currently used by theremin sensor, display controller, audio i/o, encoders interface, AXI bus interconnect and others).

I believe it's possible to get some interesting results if component parameters may slightly vary in time (not just simple tremolo).
Going to try analysis of voice, strings, wind instrument samples to extract set of sine waves modulated by frequency, phase, amplitude.
Probably, fixed f0*k base is ok if dynamic phase shift for them allows to reproduce behavior of desired instrument.

You're putting a lot of work into this, and I really appreciate that you're sharing so much of it!

I believe open sourcing of project may allow project to keep alive.

Vadim, what IDE are you using to develop your C++ and QT code?  Do you find GUI development to be difficult?

I have some programming experience in C++ and java.
The IDE I use(D) is Eclipse. Works ok.
Gui development is not difficult but there is a steep learning curve.
You can find excellent tutorials on "Cave of programming" (https://courses.caveofprogramming.com/). Some are free.
I learned my progamming skills mostly from that site. But I do not program anymore.

Vadim, what IDE are you using to develop your C++ and QT code?  Do you find GUI development to be difficult?

For Qt and even non-Qt C++ code, QtCreator is good choice. Runs on Linux as well as on Windows and Mac.
Qt uses OpenGL for hardware acceleration of rendering.

Qt makes GUI development in C++ very easy.
Look at source code of my emulator: https://github.com/fpga-theremin/theremin/tree/master/fpga/simulator/cora_theremin_sim

You can add your own widgets with a few lines of code.
E.g. in my emulator, there are encoder, tact button, register view, LCD, theremin sensor widgets.

There are no problems if you want to use your existing code written in C or C++ (e.g. emulator code).

As well, Qt provides cross-platform audio components: see audioplayer.*, audiogen.*
Feel free to reuse this code in your project (my project source is published under GPL).

I would recommend Qt + QtCreator if you want to develop cross-platform C++ GUI application.

Vadim, v7peer, thanks for the pointers!  Will give QTCreator a shot.

I installed "Geany" a couple of days ago and got my Hive sim to mostly compile, it's a lightweight and simple code / compile environment.  I'm currently struggling with getch() and kbhit() which have no clear analog in the Linux world, though I haven't tried ncurses yet.  I suppose it's due to the multi-threaded, multi-tasking environment, but getting direct control over the keyboard input and a display area shouldn't be this hard or convoluted.  And I get the legacy and all, but VT100 was like a million years ago in computer time, strange to have it looming so large over modern Linux.