The associated article seems to think that 50 steps is a long amount of time for learning this article. I would argue that 50 steps is nearly no time at all. Even though a human being may take less attempts to learn,
1) we are likely taking input from more senses than the robot is about what's occurring (we sense by stereoscopic sight, feel, etc. The robot may not sense by all of these.)
2) can apply knowledge from other domains in solving this problem (if memory serves me, this is part of the Holy Grail for artificial intelligence)
3) may make multiple attempts in our own minds before attempting to perform the activity physically again.
While I'm primarily experienced in Genetic Algorithms and NNs (so not re-enforcement learning, so much), 50 steps (or generations in a non-steady-state GA) is a very short amount of time, and so learning to properly coordinate multiple degrees of freedom into a successful activity in only 50 steps is, to me, pretty impressive.
Our biology is designed to anticipate gravity. Regardless of athletic training the average person is capable of adeptly throwing and catching a ball.
I work in construction and I regularly throw and catch objects like rolls of tape - duct tape sized rolls - 18V drills, levels, etc. but it's only when we throw something light (IE that air-resistance has an effect) and then you start getting screwed up. We don't expect things to slow down dramatically when it's thrown or falls.
We already have a substantial 'muscle memory' from doing other things with our arms as well I'd imagine...from an early age we are throwing balls, running around, picking things up.
By the end of it, you'll probably know more about the subject than I do. I'm a master's* student on my last semester. I don't actually do any work in the area, but I've taken a couple classes on neural networks and a class on genetic algorithms. Those were my major focus.
My preferred neural network structure/design is NEAT (developed by Dr. Kenneth Stanley) and its various derivatives. If I were to work more with neural networks, I would work to expand that design. (my work interests have expanded to include other stochastic algorithms, such as Monte Carlo Localization, as well)
* I was a doctoral student, but I couldn't acquire the advisor I desired (funding) and so I reduced to master's level
Comments
The associated article seems to think that 50 steps is a long amount of time for learning this article. I would argue that 50 steps is nearly no time at all. Even though a human being may take less attempts to learn, 1) we are likely taking input from more senses than the robot is about what's occurring (we sense by stereoscopic sight, feel, etc. The robot may not sense by all of these.) 2) can apply knowledge from other domains in solving this problem (if memory serves me, this is part of the Holy Grail for artificial intelligence) 3) may make multiple attempts in our own minds before attempting to perform the activity physically again.
While I'm primarily experienced in Genetic Algorithms and NNs (so not re-enforcement learning, so much), 50 steps (or generations in a non-steady-state GA) is a very short amount of time, and so learning to properly coordinate multiple degrees of freedom into a successful activity in only 50 steps is, to me, pretty impressive.
I'd say it's pretty impressive even for a human.
We also have the advantage of having some 'physics simulation' software if you will that lets us do some runs in our head before doing it physically.
Our biology is designed to anticipate gravity. Regardless of athletic training the average person is capable of adeptly throwing and catching a ball.
I work in construction and I regularly throw and catch objects like rolls of tape - duct tape sized rolls - 18V drills, levels, etc. but it's only when we throw something light (IE that air-resistance has an effect) and then you start getting screwed up. We don't expect things to slow down dramatically when it's thrown or falls.
We already have a substantial 'muscle memory' from doing other things with our arms as well I'd imagine...from an early age we are throwing balls, running around, picking things up.
Reinforcement learning was applied after the basic model was initialized with imitation. Maybe that can partially explain the small number of steps.
re: 1) Vision assisted robots can do some amazing things - http://www.hizook.com/blog/2009/08/03/high-speed-robot-hand-...
(I think re-enforcement learning was more the point than 'can we teach a robot to flip pancakes')
what sort of neuroevolution stuff do you do, i'm a phd in that area right now
By the end of it, you'll probably know more about the subject than I do. I'm a master's* student on my last semester. I don't actually do any work in the area, but I've taken a couple classes on neural networks and a class on genetic algorithms. Those were my major focus.
My preferred neural network structure/design is NEAT (developed by Dr. Kenneth Stanley) and its various derivatives. If I were to work more with neural networks, I would work to expand that design. (my work interests have expanded to include other stochastic algorithms, such as Monte Carlo Localization, as well)
* I was a doctoral student, but I couldn't acquire the advisor I desired (funding) and so I reduced to master's level