I've occasionally thought it would be fun (not necessarily productive!) to do something like the experiments folks have done with Genetic Algorithms, having simulated robots learn to walk, etc. in a simulated physics environment. The interesting bit would be to do it with real legs, sensors, servos, etc., to try to build up a "naive physics" library that can deal with balance, etc., by learning from physical experience rather than starting with equations of motion.
That may be the most number of times I've typed "etc." in a paragraph.
Can anyone explain why diffusion policies are so powerful compared to just attaching a bunch of actuators to the output layer of a deep neural network?
I remember articles about a competition DARPA had to this effect, using an off-the-shelf simulator (maybe Bullet?). At first, the simulated robots learned to hack the sim. Some did a vibration thing that messed with the friction model, sort of like a vibrating cell phone moving across a table. Others learned to hack collision detection/response, doing a sort of jump / pole-vault thing that would explosively hurl them forward. I assume they later tweaked the sim and the rewards until they actually got the walking behaviors they wanted.
I had seen that. The assumption seems to be that simulations are simplistic, and then most of what remains shows practical ways for real world training to work (while acknowledging that it's hard to do)
My university had a program that did this, I took it as a sophomore years ago while I was studying "evolutionary robotics", although typically we would simulate the robot in 3D virtual environment to test the reward mechanism for learning to walk.
System Identification is a fairly common strategy for developing a controller. Essentially, you learn the dynamics model from recorded data, and then use that as a simulator to iteratively improve the controller. The devil is in the details though, as you can run into issues with simulation fidelity (due to missing data), which can produce something that doesn't translate to real life performance.
Comments
I've occasionally thought it would be fun (not necessarily productive!) to do something like the experiments folks have done with Genetic Algorithms, having simulated robots learn to walk, etc. in a simulated physics environment. The interesting bit would be to do it with real legs, sensors, servos, etc., to try to build up a "naive physics" library that can deal with balance, etc., by learning from physical experience rather than starting with equations of motion.
That may be the most number of times I've typed "etc." in a paragraph.
You're describing something similar to what TRI has been working on for several years and seems to be making progress on: https://www.tri.global/news/toyotas-robots-are-learning-do-h...
Can anyone explain why diffusion policies are so powerful compared to just attaching a bunch of actuators to the output layer of a deep neural network?
I remember articles about a competition DARPA had to this effect, using an off-the-shelf simulator (maybe Bullet?). At first, the simulated robots learned to hack the sim. Some did a vibration thing that messed with the friction model, sort of like a vibrating cell phone moving across a table. Others learned to hack collision detection/response, doing a sort of jump / pole-vault thing that would explosively hurl them forward. I assume they later tweaked the sim and the rewards until they actually got the walking behaviors they wanted.
https://m.youtube.com/watch?v=17NrtKHdPDw
I had seen that. The assumption seems to be that simulations are simplistic, and then most of what remains shows practical ways for real world training to work (while acknowledging that it's hard to do)
My university had a program that did this, I took it as a sophomore years ago while I was studying "evolutionary robotics", although typically we would simulate the robot in 3D virtual environment to test the reward mechanism for learning to walk.
System Identification is a fairly common strategy for developing a controller. Essentially, you learn the dynamics model from recorded data, and then use that as a simulator to iteratively improve the controller. The devil is in the details though, as you can run into issues with simulation fidelity (due to missing data), which can produce something that doesn't translate to real life performance.
I've thought something similar but the reverse (if I'm understanding you correctly).
Using simulation to train a DIY spot on something novel and see how good I could get the performance in the real world.
Of course that's a dream of mine one day and not an active project.