Making synthesizers emulate the sound of real instruments was once a major goal of synthesis, but that has mostly faded outside of the rarefied world of physical modeling. Electronic musicians remain interested in sounding "warm" and "organic" (or the opposite), but it's rare to find someone who is really interested in making a synthesizer sound exactly like a guitar or a clarinet.
But when I was learning FM synthesis (one of the harder ones), I found it incredibly useful to go through the exercise of trying to make a synthesizer sound like a convincing gong, or reed instrument, or guitar string. In fact, I don't think FM really clicked until I did that.
This isn't about accurate recreations of acoustic instruments, it's about capturing some of the features of acoustic instruments in non-imitative patches to make them sound less sterile and simplistic.
A lot of (non-MIDI) computer music is either drones, granular clouds, or percussive blips.
There's an under-explored area of articulation and musical detail between those sounds and warm, mushy analog.
The DX7 was a good first take at that. But today you can have hundreds of oscillators stacked and patched in any configuration you want instead of just six.
You can actually use thousands of oscillators, which is what Magenta did with their differentiable DSP (DDSP) approach to modeling acoustic instruments including singing:
Feels like some better instruments can help with this, over the basic keyboard. Things with polyphonic aftertouch, or electronic wind instruments, or more advanced surfaces like the Linnstrument or Continuum.
"it's rare to find someone who is really interested in making a synthesizer sound exactly like a guitar or a clarinet"
Perhaps that's because it's quite difficult to do that with synthesizers? Because once you start using samples, you loss the ability to play and modulate without it sounding like a sample?
I guess my counterpoint would be the massive popularity of Omnisphere, which combines a sophisticated sample-based synthesizer with good filters and a few other types of synthesis. Or would you consider this "organic" but not "realistic/acoustic"?
Many synthesizers do not use samples. Physical modeling oscillators have been available in good synthesis engines for a long time. These generate sounds based on a physics model of real acoustic instrument designs and materials, dynamically reacting like real instruments. These used to be dedicated synthesizers but are now buried in a long list of oscillator models available on general purpose synthesizers.
I used to own a Korg Z1, which had one of the first physical modeling synthesis engines. While it could produce relatively realistic acoustic sounds, that is not what people ended up using these types of synthesizers for. Unlike a physical acoustic instrument, where the basic geometry, material properties, and other parameters are fixed, most parameters of the synthesized physical models could be dynamically modulated and manipulated like any other synthesis parameter.
Modulating the fundamental physics properties of an acoustic instrument at audio rates produces some really interesting timbres and effects that are not reproducible using any other type of synthesis or (obviously) acoustic instruments. Consequently, no one ended up using them for realistic acoustic sounds; it was much more interesting to use the synthesis engine to do physically impossible manipulation of the acoustic model to generate novel sounds.
These days almost everything in a synthesizer engine is based on modeling due to the inexorable increase in available computing power. Nonetheless, there is still considerable convenience and economy in using acoustic instruments or samples for many purposes. Just because we can create incredibly detailed and realistic physical models doesn't mean it is worth the effort and they often have terrible UI. I could see this as being something where AI could do a lot.
Omnisphere is a good example of where the term "sample-based" isn't entirely accurate (ignoring its modeled oscillators). While it does have a large and excellent sample library, many of the oscillator engines use the samples as spectral feedstock instead of as a sound to be played per se. The raw sample is not identifiable even though it imparts a characteristic quality on how the oscillator sounds. (Omnisphere is also massively popular because it is an excellent synthesis engine with an unusually good ease-of-use to power ratio. Still one of my all-time favorites.)
Problem: We don't really know what "realistic" quality means. Is it
the timbre? Is it the spectral dynamics? Is it a set of recognisable
behaviours? Does playing versus only listening to a simulated
instrument make a difference to the perception of "realism"? Sure we
can do naive A-B tests on audio, but they turn out to show unexpected
or "wrong" results.
Not that I'd encourage anyone to get involved in academia in its
current degenerate state, but for someone really passionate about this
I'd say this postdoc Diemo is heading at IRCAM looks like one of the
more fun, interesting and challenging projects around right now. FWIW
there's a similar programme running at Edinburgh with Stefan Bilbao
and Rod Selfridge testing quality of PDE synthesis and AI discovery of
parametric control.
High end realistic patches allow you to move mics around the virtual space, change mics, move musicians, control airflow, bowing speed and pressure etc. They sound REALLY good these days, almost indistinguishable from the real thing.
I think this article is more geared towards pure synthesis when you are using very few primary harmonics. Like stacked sine waves or something. They are impossible to eq because the frequencies are too pure and unnatural so you have to dirty them up.
Sample an elephant, synplant to synthesize it. As a non-singer synplant has allowed me to be on my tracks...you could never tell. Also, amseveral nembers of my friends and family are a) knocked out by the simple fun of synplant, and hearing/ looking at what ai believes constitutes their voices.
Not for DX7. It creates patches for it's own synth (2 oscillators, FM, envelopes, LFO, and effects). You can then look at how they are build and edit the parameters.
Just to give an example of "the rarefied field of physical modeling": SampleModeling/AudioModeling/SWAM instruments enjoy some popularity, I'd say they were the go-to demo when MPE controllers had a burst in popularity a couple of years ago. Most of them use a hybrid sampling-synthesis approach, and some of them use almost entirely synthesis. Astoundingly realistic when used correctly.
Personally, I'd say physical modeling still has a lot of unexplored potential, but that doesn't seem to be a popular opinion.
Agree with you there on both counts. Attempting to match physical instruments is a great learning exercise, but not usually that interesting creatively. However patches that live someone in the middle ground between naturalistic and synthetic are often the most interesting IMO.
i think its because sample based virtual instruments, while enormous in hard drive space, do a good job of approximating many classical instruments and have advanced enough to be useful without being difficult to implement.
I could learn a new synth or I could use the Spitfire Labs BBC Symphony Orchestra pack and spend a lot of takes playing with macro wheels and maybe some adjusting after to try to improve articulation to something more natural sounding. It would by a bear live but in the studio its easy.|
The synth would offer more customization but the sample one is good enough for most things, especially in a small/home studio setting.
Synth strings can be great, but in their own aesthetics. I'd argue the sample pack is much, much more realistic and rich sounding than any synth patch you can come up with. There's no easily programmable synth that can emulate the mixture of a dozen or so strings vibrating at slightly different pitches. Creating a single, somewhat realistic violin is almost impossible outside of very large physical models (there's a good model in Reaktor by Chet Singer). Even dedicated physical modelling synths fail at that task.
I'm trying to make FM synthesis mimic a particular sound but I find it very difficult to search for something that even approximates the sound I'm looking to emulate (an elephant's roar). Do you have any tips?
Edit - from the comments I found this paper which looks promising
Comments
Making synthesizers emulate the sound of real instruments was once a major goal of synthesis, but that has mostly faded outside of the rarefied world of physical modeling. Electronic musicians remain interested in sounding "warm" and "organic" (or the opposite), but it's rare to find someone who is really interested in making a synthesizer sound exactly like a guitar or a clarinet.
But when I was learning FM synthesis (one of the harder ones), I found it incredibly useful to go through the exercise of trying to make a synthesizer sound like a convincing gong, or reed instrument, or guitar string. In fact, I don't think FM really clicked until I did that.
This isn't about accurate recreations of acoustic instruments, it's about capturing some of the features of acoustic instruments in non-imitative patches to make them sound less sterile and simplistic.
A lot of (non-MIDI) computer music is either drones, granular clouds, or percussive blips.
There's an under-explored area of articulation and musical detail between those sounds and warm, mushy analog.
The DX7 was a good first take at that. But today you can have hundreds of oscillators stacked and patched in any configuration you want instead of just six.
You can actually use thousands of oscillators, which is what Magenta did with their differentiable DSP (DDSP) approach to modeling acoustic instruments including singing:
https://magenta.tensorflow.org/ddsp-vst
https://magenta.tensorflow.org/ddsp
I doesn't sound realistic but it is expressive in an unusual way.
Feels like some better instruments can help with this, over the basic keyboard. Things with polyphonic aftertouch, or electronic wind instruments, or more advanced surfaces like the Linnstrument or Continuum.
"it's rare to find someone who is really interested in making a synthesizer sound exactly like a guitar or a clarinet"
Perhaps that's because it's quite difficult to do that with synthesizers? Because once you start using samples, you loss the ability to play and modulate without it sounding like a sample?
I guess my counterpoint would be the massive popularity of Omnisphere, which combines a sophisticated sample-based synthesizer with good filters and a few other types of synthesis. Or would you consider this "organic" but not "realistic/acoustic"?
Many synthesizers do not use samples. Physical modeling oscillators have been available in good synthesis engines for a long time. These generate sounds based on a physics model of real acoustic instrument designs and materials, dynamically reacting like real instruments. These used to be dedicated synthesizers but are now buried in a long list of oscillator models available on general purpose synthesizers.
I used to own a Korg Z1, which had one of the first physical modeling synthesis engines. While it could produce relatively realistic acoustic sounds, that is not what people ended up using these types of synthesizers for. Unlike a physical acoustic instrument, where the basic geometry, material properties, and other parameters are fixed, most parameters of the synthesized physical models could be dynamically modulated and manipulated like any other synthesis parameter.
Modulating the fundamental physics properties of an acoustic instrument at audio rates produces some really interesting timbres and effects that are not reproducible using any other type of synthesis or (obviously) acoustic instruments. Consequently, no one ended up using them for realistic acoustic sounds; it was much more interesting to use the synthesis engine to do physically impossible manipulation of the acoustic model to generate novel sounds.
These days almost everything in a synthesizer engine is based on modeling due to the inexorable increase in available computing power. Nonetheless, there is still considerable convenience and economy in using acoustic instruments or samples for many purposes. Just because we can create incredibly detailed and realistic physical models doesn't mean it is worth the effort and they often have terrible UI. I could see this as being something where AI could do a lot.
Omnisphere is a good example of where the term "sample-based" isn't entirely accurate (ignoring its modeled oscillators). While it does have a large and excellent sample library, many of the oscillator engines use the samples as spectral feedstock instead of as a sound to be played per se. The raw sample is not identifiable even though it imparts a characteristic quality on how the oscillator sounds. (Omnisphere is also massively popular because it is an excellent synthesis engine with an unusually good ease-of-use to power ratio. Still one of my all-time favorites.)
Seems like a good opportunity to share this [0]
Problem: We don't really know what "realistic" quality means. Is it the timbre? Is it the spectral dynamics? Is it a set of recognisable behaviours? Does playing versus only listening to a simulated instrument make a difference to the perception of "realism"? Sure we can do naive A-B tests on audio, but they turn out to show unexpected or "wrong" results.
Not that I'd encourage anyone to get involved in academia in its current degenerate state, but for someone really passionate about this I'd say this postdoc Diemo is heading at IRCAM looks like one of the more fun, interesting and challenging projects around right now. FWIW there's a similar programme running at Edinburgh with Stefan Bilbao and Rod Selfridge testing quality of PDE synthesis and AI discovery of parametric control.
[0] https://www.ircam.fr/job-offer/chargee-de-recherche-projet-a...
High end realistic patches allow you to move mics around the virtual space, change mics, move musicians, control airflow, bowing speed and pressure etc. They sound REALLY good these days, almost indistinguishable from the real thing.
I think this article is more geared towards pure synthesis when you are using very few primary harmonics. Like stacked sine waves or something. They are impossible to eq because the frequencies are too pure and unnatural so you have to dirty them up.
Synplant 2 can now do that automatically. It uses AI to create synth patches that emulate the input sample.
Sample an elephant, synplant to synthesize it. As a non-singer synplant has allowed me to be on my tracks...you could never tell. Also, amseveral nembers of my friends and family are a) knocked out by the simple fun of synplant, and hearing/ looking at what ai believes constitutes their voices.
Can it create FM synth patches in particular? Like a patch for a DX7.
I have a project that aims to do the similar task but with a different approach:
https://github.com/chaosprint/RaveForce
not finished yet, so pr is welcomed!
Not for DX7. It creates patches for it's own synth (2 oscillators, FM, envelopes, LFO, and effects). You can then look at how they are build and edit the parameters.
Aphex Twin tool for that: https://magpi.raspberrypi.com/articles/aphex-twin-midimutant
Just to give an example of "the rarefied field of physical modeling": SampleModeling/AudioModeling/SWAM instruments enjoy some popularity, I'd say they were the go-to demo when MPE controllers had a burst in popularity a couple of years ago. Most of them use a hybrid sampling-synthesis approach, and some of them use almost entirely synthesis. Astoundingly realistic when used correctly.
Personally, I'd say physical modeling still has a lot of unexplored potential, but that doesn't seem to be a popular opinion.
Agree with you there on both counts. Attempting to match physical instruments is a great learning exercise, but not usually that interesting creatively. However patches that live someone in the middle ground between naturalistic and synthetic are often the most interesting IMO.
i think its because sample based virtual instruments, while enormous in hard drive space, do a good job of approximating many classical instruments and have advanced enough to be useful without being difficult to implement.
I could learn a new synth or I could use the Spitfire Labs BBC Symphony Orchestra pack and spend a lot of takes playing with macro wheels and maybe some adjusting after to try to improve articulation to something more natural sounding. It would by a bear live but in the studio its easy.|
The synth would offer more customization but the sample one is good enough for most things, especially in a small/home studio setting.
Synth strings can be great, but in their own aesthetics. I'd argue the sample pack is much, much more realistic and rich sounding than any synth patch you can come up with. There's no easily programmable synth that can emulate the mixture of a dozen or so strings vibrating at slightly different pitches. Creating a single, somewhat realistic violin is almost impossible outside of very large physical models (there's a good model in Reaktor by Chet Singer). Even dedicated physical modelling synths fail at that task.
I'm trying to make FM synthesis mimic a particular sound but I find it very difficult to search for something that even approximates the sound I'm looking to emulate (an elephant's roar). Do you have any tips?
Edit - from the comments I found this paper which looks promising
https://ccrma.stanford.edu/~cc/vox/smac2013som/upho2013.pdf
Which also leads me to this paper by Chowning himeself from 1980 titled 'Computer synthesis of the singing voice' which uses FM synthesis.
https://www.speech.kth.se/music/publications/kma/papers/kma2...
I think Roland's digital piano sounds are entirely simulated/modelled, they don't use sampling like for example Yamaha does for their digital pianos.
However, is that a continuation of synthesizers or not?