For instance in robotics, the inverse kinematics could be formulated using the well-known equations, and combined with a CNN for vision, maybe do reinforcement learning across the whole thing. Or in audio classification, could use standard filterbanks construction for creating a time-frequency (spectrogram) representation, then combined with a CNN for classification.
Right now these things are done either:
1) Unstructured end-2-end learning, where a deep neural network has to discover the laws of physics unaided by structure. The models are massively over-parametrized, require a lot of data to train and vulnerable to adverse inputs.
2) As independent systems, optimized separately. If the first model is complex, the model that follows must usually reflect this complexity. A simpler global solution might exist, but cannot be found.
It is however super early days. Right now there are hints that going in this direction might be fruitful, but as far as I know, not many concrete wins or a lot of practice. That will take some years still.
Comments
For instance in robotics, the inverse kinematics could be formulated using the well-known equations, and combined with a CNN for vision, maybe do reinforcement learning across the whole thing. Or in audio classification, could use standard filterbanks construction for creating a time-frequency (spectrogram) representation, then combined with a CNN for classification.
Right now these things are done either: 1) Unstructured end-2-end learning, where a deep neural network has to discover the laws of physics unaided by structure. The models are massively over-parametrized, require a lot of data to train and vulnerable to adverse inputs. 2) As independent systems, optimized separately. If the first model is complex, the model that follows must usually reflect this complexity. A simpler global solution might exist, but cannot be found.
It is however super early days. Right now there are hints that going in this direction might be fruitful, but as far as I know, not many concrete wins or a lot of practice. That will take some years still.