jimmySixDOF on Gsplat: CUDA accelerated rasterization of gaussians with Python bindings

Comment on Gsplat: CUDA accelerated rasterization of gaussians with Python bindings

There is an interesting problem being worked on with Gaussian Splats for use in VR stereo rendering [1]

Gaussian splats are 3d, but currently rendered as 2d cards ordered on centre, not per pixel like they should be.

So as centres reorder, they "shuffle". This is a showstopper for general adoption in VR

The solution? Ordering per pixel will solve this, as would stochastic methods which write depth correctly (though those have other problems for VR).

For it to be viable you need to cull much more aggressively but also can't have any warp divergence when processing samples.

Unforunately those things are in direct conflict. Paper's renderer is at extreme end of "weak cull, perfect warp convergence".

Apparently need some new Order-Independent Transparency algorithms that handle depth layers.

[1] https://x.com/charshenton/status/1710207169407447396?s=20

teucris 2y

I hacked in a weighted stochastic method into a WebGPU solution[0] and it worked surprisingly well. Essentially, given each pixel of a splat, pick a pseudo-random number from zero to one and if it’s lower than the alpha value for the pixel, draw over whatever is in the frame buffer at full opacity.

The next step I wanted to try was to super-sample (eg render at double resolution) and linearly down sample. I imagine this would give a slightly noisy result that I think would be pretty visually pleasing.

0: https://github.com/antimatter15/splat

Comments