Those are amazing accomplishments but I am more interested in research developments in things like In-Memory (Analog) or other different approaches.
Companies like EnCharge, Mythic, etc.
And much more efficient devices like RRAM, MRAM, and FETs. Like FE-FETs with AlScN.
The stuff just coming out of research or still in research is more exciting in terms of the potential for truly huge efficiency and performance boosts.
Taalas is interesting also because of it's efficiency and speed. Guess it was just purchased by AMD.
I'm not sure that more efficient compute can win out, since it also has to beat out the competition on capital investment. There's an upper bound of 'just get 3 sets of GPUs and run them on solar in three different timezones' where the cost of electricity is negligible. Efficient digital architectures run about 1/5 or 1/10th the speed and require much more wafer area. Analog technologies seem unlikely to have the fidelity required for training, though inference sounds plausible. It's also not clear that analog technologies would necessarily use less power.
Transistors are most power efficient when they are either completely open (no resistance) or completely closed (no current).
If they are in some analog half-open state they have both resistance and current, which means waste heat.
Also transistors in modern chips are already so small that you can literally count the number of electrons flowing through them. At the smallest scale, nature becomes discrete. So if there are something like 128 electrons, you only get 7 bits of 'analog' to play with anyway.
Recently I started thinking about an analog optical device capable of matrix multiplication. After some online research, the Mach-Zehnder interferometer seems to have been adapted by a German company into a full addon optical based matrix multiplication system.
It is a very interesting concept and the company Q.Ant has delivered working products and software. Apparently it can greatly speed up certain training applications and future designs will solve some of the current problems.
The energy and performance advantages seem to be worth the R&D. If some form of optical memory to replace DRAM in the chain could be designed, that would solve several conversion costs between optical to digital.
Either way it is pretty cool to do math this way, basically just using light.
The assumption that optical or analog will be more efficient for a given accuracy though is not grounded. Any non-linear optical phenomenon requires multiple photons interacting in matter, or interacting sequentially with the same matter, and this requires high intensities of (almost always coherent) light in order to make the interaction happen frequently enough.
e.g for a Mach-Zehnder interferometer measuring the phase shifts requires many photons. A single photon arrives or doesn't arrive at the detector. Even if you could perfectly ensure it hits detector at the same time as the reference photon and subsequently detect with 100% quantum efficiency, that still only gives you 1 bit of phase information, not an analog value.
Digital can be done more efficiently with adiabatic or asynchronous circuits. The limiting factor then is speed and chip area rather than joules-per-and-gate.
Some technologies will definitely be much faster and much more power efficient. They haven't come out of research or if they have they have not been scaled yet, so it's not 100% proven. But it looks likely for multiple options to get there.
Agreed. One of the least efficient things to do with digital logic is floating point multiplication. The second least efficient is probably integer multiply (I'm just kidding a bit here).
Multiplication may make sense for the math, but it's just not practical in hardware. There has to be a better, much simpler operation, that can give you equivalent results. Yes, you may have to re-think and re-train everything, but eventually it should make current architectures obsolete.
Comments
Those are amazing accomplishments but I am more interested in research developments in things like In-Memory (Analog) or other different approaches.
Companies like EnCharge, Mythic, etc.
And much more efficient devices like RRAM, MRAM, and FETs. Like FE-FETs with AlScN.
The stuff just coming out of research or still in research is more exciting in terms of the potential for truly huge efficiency and performance boosts.
Taalas is interesting also because of it's efficiency and speed. Guess it was just purchased by AMD.
I'm not sure that more efficient compute can win out, since it also has to beat out the competition on capital investment. There's an upper bound of 'just get 3 sets of GPUs and run them on solar in three different timezones' where the cost of electricity is negligible. Efficient digital architectures run about 1/5 or 1/10th the speed and require much more wafer area. Analog technologies seem unlikely to have the fidelity required for training, though inference sounds plausible. It's also not clear that analog technologies would necessarily use less power.
Transistors are most power efficient when they are either completely open (no resistance) or completely closed (no current).
If they are in some analog half-open state they have both resistance and current, which means waste heat.
Also transistors in modern chips are already so small that you can literally count the number of electrons flowing through them. At the smallest scale, nature becomes discrete. So if there are something like 128 electrons, you only get 7 bits of 'analog' to play with anyway.
Recently I started thinking about an analog optical device capable of matrix multiplication. After some online research, the Mach-Zehnder interferometer seems to have been adapted by a German company into a full addon optical based matrix multiplication system.
It is a very interesting concept and the company Q.Ant has delivered working products and software. Apparently it can greatly speed up certain training applications and future designs will solve some of the current problems.
The energy and performance advantages seem to be worth the R&D. If some form of optical memory to replace DRAM in the chain could be designed, that would solve several conversion costs between optical to digital.
Either way it is pretty cool to do math this way, basically just using light.
The assumption that optical or analog will be more efficient for a given accuracy though is not grounded. Any non-linear optical phenomenon requires multiple photons interacting in matter, or interacting sequentially with the same matter, and this requires high intensities of (almost always coherent) light in order to make the interaction happen frequently enough.
e.g for a Mach-Zehnder interferometer measuring the phase shifts requires many photons. A single photon arrives or doesn't arrive at the detector. Even if you could perfectly ensure it hits detector at the same time as the reference photon and subsequently detect with 100% quantum efficiency, that still only gives you 1 bit of phase information, not an analog value.
Digital can be done more efficiently with adiabatic or asynchronous circuits. The limiting factor then is speed and chip area rather than joules-per-and-gate.
Some technologies will definitely be much faster and much more power efficient. They haven't come out of research or if they have they have not been scaled yet, so it's not 100% proven. But it looks likely for multiple options to get there.
Recently on HN: https://news.ycombinator.com/item?id=49352470
Agreed. One of the least efficient things to do with digital logic is floating point multiplication. The second least efficient is probably integer multiply (I'm just kidding a bit here).
Multiplication may make sense for the math, but it's just not practical in hardware. There has to be a better, much simpler operation, that can give you equivalent results. Yes, you may have to re-think and re-train everything, but eventually it should make current architectures obsolete.
memrystors for chache inside the pipeline