For instance, we solve nontrivial nonlinear PDEs (one HJB equation and one Black-Scholes equation) in 100,000 dimensions in 6 hours on a single GPU using SDGD with PINNs
The way ML people use "dimension", each free parameter is an extra dimension. A high-resolution 2D image is considered to have millions of dimensions - one for each pixel.
That’s not just ML, that’s linear algebra. Moreover, across fields in math, the dimension of an object is defined differently, but it’s always a very fundamental property of an object that captures loosely how complex an object is. Often, that complexity is related to how many numbers you need to describe said object.
These are all expressions of the same concept. The confusion is that physicists are describing the dimensionality of specific systems (space, spacetime, superstring theory, supergravity) - that doesn't mean that this limits the dimensionality of other systems (which is often where confusion lies between laymen).
We usually do HxWxC, for height, width, and channels, so each pixel is addressed via the two first dims of the input, and then it has 3 channels. Of course, you can transpose the tensor to CxHxW or CxWxH. Different ordering behaves differently with respect to memory locality.
Physics uses vectors, which are multi-dimensional and can represent a lot more than 11 dimensions... Machine learning typically uses feature vectors, which are basically just lists of numerical properties.
To add to the other answers: dimensions also appear in higher-order PDE: you typically go from an nth order 1-dimensional equation (an equation mentioning the n-th derivative of your unknown function) to an n-dimensional 1st order equation. There is a general pattern of adding parameters to have several simple problems instead of one complicated.
Comments
100,000 dimensions? I thought there were like... 11 tops? https://imagine.gsfc.nasa.gov/science/questions/superstring....
(edit: oops, I misunderstood the context of dimensions here. my bad. thanks)
The way ML people use "dimension", each free parameter is an extra dimension. A high-resolution 2D image is considered to have millions of dimensions - one for each pixel.
That’s not just ML, that’s linear algebra. Moreover, across fields in math, the dimension of an object is defined differently, but it’s always a very fundamental property of an object that captures loosely how complex an object is. Often, that complexity is related to how many numbers you need to describe said object.
These are all expressions of the same concept. The confusion is that physicists are describing the dimensionality of specific systems (space, spacetime, superstring theory, supergravity) - that doesn't mean that this limits the dimensionality of other systems (which is often where confusion lies between laymen).
And each pixel is what, a RGB-vector?
We usually do HxWxC, for height, width, and channels, so each pixel is addressed via the two first dims of the input, and then it has 3 channels. Of course, you can transpose the tensor to CxHxW or CxWxH. Different ordering behaves differently with respect to memory locality.
In the context of neural networks, kind of. It’s 3 numbers on the input layer, but that could influence N parameters in later layers.
Yes, each pixel is a vector of the rasterized vector space.
Thanks for clarifying the dimensions.
Physics uses vectors, which are multi-dimensional and can represent a lot more than 11 dimensions... Machine learning typically uses feature vectors, which are basically just lists of numerical properties.
To add to the other answers: dimensions also appear in higher-order PDE: you typically go from an nth order 1-dimensional equation (an equation mentioning the n-th derivative of your unknown function) to an n-dimensional 1st order equation. There is a general pattern of adding parameters to have several simple problems instead of one complicated.