In pinhole cameras, straight lines look straight. That's what a regular projection matrix gives you with rasterization. With non-pinhole cameras, straight lines look curved. You can't rasterize this directly. 3D Gaussian splats have an issue with this, addressed by methods like ray tracing. It's very useful to train on non-pinhole cameras, because in real world they can capture a wider field of view.