People interested in trying out this kind of thing should also check out vcvrack. Its a free fully featured modular synthesizer system, also very easy to set up. It should be possible to run it on a pi, though i havent tried that personally. It also has excelent midi integration, so one could try to reproduce some of the hands free control with a program that does voice recognition and supports a virtual midi interface.