If it's still retrieving an image and not reconstructing it, if the dataset is large enough that's decently fine, but this is generally not how diffusion models work in general and I'd have expected the model to map the fMRI data to a wholly new image.
Comments
If it's still retrieving an image and not reconstructing it, if the dataset is large enough that's decently fine, but this is generally not how diffusion models work in general and I'd have expected the model to map the fMRI data to a wholly new image.
Please read the paper. Or at least the blog post. It's really quite readable.
They explain that they've done both retrieval and reconstruction, and have lots of pictures showing examples of each.
https://medarc-ai.github.io/mindeye/
If you can retrieve an image using a latent vector, it’s trivial to reconstruct it (decently well) with a diffusion model.
They tested themselves both on retrieval and reconstruction.