The next step is to automatically add "nodes" to the 3D images where the model can pivot, rotate and whatnot and then boom, you have on-demand animated, interactive content

My gut says a 3D engine + this would be a superior solution to the current approach of rendering rasterized video directly from the latents (coincidentally, Sora got released today).

It may not be tractable to train a network to rig and animate meshes, as well as setting up an entire scene to be a "digital twin" of random videos, bit I imagine such a set up would have finer-grained control over the created video while keeping everything else in it the unchanged