Well.. autodiff isn't particularly technically sophisticated. But I doubt that when it was first invented in the 1950s, people could have foreseen that a lot of autodiff put together can write an implementation of autodiff.
If you're interested in more technical approaches, I think they're starting to come together, slowly. I've seen several research directions (operads / cellular sheaves for a theory of multiagent collaboration, the theory of open games for compositional forwards/backwards strategy/planning in open environments) that would fit in quite nicely. And to some extent, the earliest frameworks are going to be naive implementations of these directions.
Comments
Well.. autodiff isn't particularly technically sophisticated. But I doubt that when it was first invented in the 1950s, people could have foreseen that a lot of autodiff put together can write an implementation of autodiff.
If you're interested in more technical approaches, I think they're starting to come together, slowly. I've seen several research directions (operads / cellular sheaves for a theory of multiagent collaboration, the theory of open games for compositional forwards/backwards strategy/planning in open environments) that would fit in quite nicely. And to some extent, the earliest frameworks are going to be naive implementations of these directions.