For years Google TPUs have been boring, presumably optimizing TCO for pre-existing workloads. Now Google is in an existential crisis where it may be winner-take-all, so it makes sense to burn money to copy NVIDIA: spend more money on bigger chips, high memory bandwidth, better process nodes (but not thee best), and fast pods. It may not be enough.
Comments
For years Google TPUs have been boring, presumably optimizing TCO for pre-existing workloads. Now Google is in an existential crisis where it may be winner-take-all, so it makes sense to burn money to copy NVIDIA: spend more money on bigger chips, high memory bandwidth, better process nodes (but not thee best), and fast pods. It may not be enough.