"Small" is 32b a9b for 19GB @ Q4_K_XL

20GB @ 100,000 context.

But for some reason... LM studio isnt loading it onto gpu for me?

I just updated to 0.3.28 and still wont load onto gpu.

Switching from Vulkan to rocm. It's now working properly?

https://docs.unsloth.ai/new/ibm-granite-4.0

Fantastic work from unsloth folks as usual.

As it's running in roo code, it's using more like 26GB of vram.

~30TPS

Roo code does not work with it.

Kilo code next. It seems to be about 22GB of vram.

Kilo code works great.

The model however didn't 1 shot my first benchmark. That's pretty bad news for this model given magistral 2509 or apriel 15b are better.

Better on pass 2, still no 100%

3rd pass achieved.

Im predicting it'll be around 30% on livecodebench. Probably like 15% on aiderpolyglot. Very disappointed in its coding capability.

I just found:

https://artificialanalysis.ai/models/granite-4-0-h-small

25.1% on livecodebench. Absolutely deserved.

2% terminal bench.

16% on coding index. Completely deserved.