So you can't blame the PyTorch team. If there's anyone to blame, it's Google Cloud. In the meantime, I don't think there's any harm advertising PyTorch with TPU support if running on TPUs with PyTorch is often much faster than running on GPUs with PyTorch.
In the above-linked GitHub issue, the Google TPU team is now giving an ETA of early 2021. At that point PyTorch TPU training (including on TPU pods) should be equivalent to TF TPU training. But I think my point still stands that as long as PyTorch TPU training is faster than GPU training, even in its current state, there's nothing wrong advertising TPU support now.
Comments
We both know (and correct me if I'm wrong) that this is an issue with the GCP cloud architecture. https://github.com/pytorch/xla/issues/1858
So you can't blame the PyTorch team. If there's anyone to blame, it's Google Cloud. In the meantime, I don't think there's any harm advertising PyTorch with TPU support if running on TPUs with PyTorch is often much faster than running on GPUs with PyTorch.
In the above-linked GitHub issue, the Google TPU team is now giving an ETA of early 2021. At that point PyTorch TPU training (including on TPU pods) should be equivalent to TF TPU training. But I think my point still stands that as long as PyTorch TPU training is faster than GPU training, even in its current state, there's nothing wrong advertising TPU support now.