I was thinking about this approach, and this paper basically validates it.
The inference cost must be extremely high compared to diffusion based approaches. I wonder if this would ever be useful for more organic sculpting type workflows. E.g. for organic non-hard surface models, use a diffusion model for generation, and leverage this LLM codegen and tool call approach for retopology and cleanup.
My belief is it'll get better over time with organic shapes as LLMs improve their ability to synthesize higher order differentials. I also believe the future may not be coded 3D everywhere, but a mix of code and dumb 3D. But I'm not a huge fan of inverse code recovery; I found that to be extremely lossy.
Do you want to connect over discord or linkedin or something, to cross-pollinate ideas?
Comments
I was thinking about this approach, and this paper basically validates it.
The inference cost must be extremely high compared to diffusion based approaches. I wonder if this would ever be useful for more organic sculpting type workflows. E.g. for organic non-hard surface models, use a diffusion model for generation, and leverage this LLM codegen and tool call approach for retopology and cleanup.
My belief is it'll get better over time with organic shapes as LLMs improve their ability to synthesize higher order differentials. I also believe the future may not be coded 3D everywhere, but a mix of code and dumb 3D. But I'm not a huge fan of inverse code recovery; I found that to be extremely lossy.
Do you want to connect over discord or linkedin or something, to cross-pollinate ideas?