Skip to content

Comment on MLC-LLM: GPT/Llama on consumer-class GPUs and phonesparent

Comments

Sure, but other than Apple no one has the tight integration needed to make that happen. If Apple leans into their CoreML stuff they have a real opportunity to steal the market from Nvidia.

Long term maybe Nvidia is able to release release some integrated ARM chip, but I’m not holding my breath.

Nvidia sells data center products, Apple is not going to steal this market, they aren't even in it. The gaming market is dominated by Windows, I don't see this moving meaningfully just because apple has some marginally better hardware at this point in time.

The Risc V area, now there we can talk about disruption long term

How do you imagine consoles work? And what about Apple's tight integration has actually made itself useful? The developer experience is terrible.

It’s useful because you can buy a Macbook with 64GB of RAM, and then use that RAM as VRAM to run your LLM.

NVidia Jetson is a line of products with tight integration between CPU and GPU. As just one example.

nl

Does it? I'm not aware of any way to share main memory with the Jetson CUDA cores.

If I'm wrong I'd love a pointer to the docs about it!

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.