For anyone else outside of machine learning who was wondering what all of this is, here is my best explanation:
The inferencing phase of a neural network attempts to minimize error or loss as defined by the user. This is done by iteratively applying gradient descent to the error function. Thus, the error function must have a known derivative, which can be difficult if the function has loops and conditionals.
Autograd is a software package that produces derivatives of a function automatically. It creates a computation graph of the user-defined code from which it can determine the derivative:
XLA is a JIT from TensorFlow that compiles common array functions. The JAX project from this GitHub page brings JIT optimizations to Autograd's automatic differentiation. That will speed-up the error function when inferring a neural network. Neat!
(I would be grateful for any corrections to my above explanation as I am not an expert in ML.)
Another small correction: every instance of "inference" in your comment should probably be replaced with "training." It's the training phase the involves running gradient descent of various flavors to optimize the network parameters.
It's from statistical inference, e.g., when the goal is to find the values of a model's parameters that match the sample. So if the model is y = f(x, params), inference gives you params, and prediction gives you y for a value of x that you haven't seen before.
More than that, inference usually refers to acts of decision making or evidence evaluation, like testing hypotheses or interpreting confidence/credible intervals.
Small correction: XLA is a compiler in the more general sense, not a JIT specifically. (It's a domain-specific compiler focused on linear algebra.). JAX uses XLA to do JIT compilation.
I think this is more exploiting XLA to speedup autograd than for deep learning. You would generally use tensorflow for actual training, I've never encountered any situations where autograd had to be used during training.
Comments
For anyone else outside of machine learning who was wondering what all of this is, here is my best explanation:
The inferencing phase of a neural network attempts to minimize error or loss as defined by the user. This is done by iteratively applying gradient descent to the error function. Thus, the error function must have a known derivative, which can be difficult if the function has loops and conditionals.
Autograd is a software package that produces derivatives of a function automatically. It creates a computation graph of the user-defined code from which it can determine the derivative:
https://en.wikipedia.org/wiki/Automatic_differentiation
XLA is a JIT from TensorFlow that compiles common array functions. The JAX project from this GitHub page brings JIT optimizations to Autograd's automatic differentiation. That will speed-up the error function when inferring a neural network. Neat!
(I would be grateful for any corrections to my above explanation as I am not an expert in ML.)
Another small correction: every instance of "inference" in your comment should probably be replaced with "training." It's the training phase the involves running gradient descent of various flavors to optimize the network parameters.
This is an important point.
Inferencing means “to predict” (I’m not sure when this terminology became popular; a few years ago most of us were just using the word predict)
Once trained, a model no longer requires derivatives. It’s more or less a function evaluation, which can be done on plain CPUs.
It's from statistical inference, e.g., when the goal is to find the values of a model's parameters that match the sample. So if the model is y = f(x, params), inference gives you params, and prediction gives you y for a value of x that you haven't seen before.
Also, shouldn't the verb be inferring?
More than that, inference usually refers to acts of decision making or evidence evaluation, like testing hypotheses or interpreting confidence/credible intervals.
Except in AI terminology[1], it seems that inference doesn't mean outputting params. It means outputting y.
Yes, I believe the verb is "inferring".
[1] https://blogs.nvidia.com/blog/2016/08/22/difference-deep-lea...
You're probably right then. I find AI nomenclature to be a bit of a mess.
Small correction: XLA is a compiler in the more general sense, not a JIT specifically. (It's a domain-specific compiler focused on linear algebra.). JAX uses XLA to do JIT compilation.
I think this is more exploiting XLA to speedup autograd than for deep learning. You would generally use tensorflow for actual training, I've never encountered any situations where autograd had to be used during training.