Comment on Moebius: 0.2B image inpainting model with 10B-level performanceparentComments−g588928812mowell done!unet weights are in fp32. did you by any chance try something lower, fp16?−da_grift_shift2moThe model considered it.There are 25 or so mentions of fp16 and fp32 weights across the 7500+ words of Markdown text it generated. So the next question might be: Did it make the right calls?https://github.com/simonw/moebius-web/blob/main/notes.mdhttps://github.com/simonw/moebius-web/blob/main/plan.mdhttps://github.com/simonw/moebius-web/blob/main/research.mdhttps://github.com/simonw/moebius-web/blob/main/understandin...
Comments
well done!
unet weights are in fp32. did you by any chance try something lower, fp16?
The model considered it.
There are 25 or so mentions of fp16 and fp32 weights across the 7500+ words of Markdown text it generated. So the next question might be: Did it make the right calls?
https://github.com/simonw/moebius-web/blob/main/notes.md
https://github.com/simonw/moebius-web/blob/main/plan.md
https://github.com/simonw/moebius-web/blob/main/research.md
https://github.com/simonw/moebius-web/blob/main/understandin...