That already exists though. I believe Braindance VR uses a rig with a couple dozen cameras to capture the same scene from multiple viewpoints then converts it to a gaussian splat that can be walked around.