I cannot imagine local models running on an igpu could get anything close to either a useful plan or execution of a plan. I've tested Qwen 3.5 27B locally and its solutions to coding problems are usually flawed and running in thinking mode is too slow and that's on a discrete GPU. How do you get anything useful done on a model that runs on an igpu?
yeah, i dont understand, whatsoever, the folks who are running models locally other than for privacy/curiosity. It is surely FAR slower, more expensive (electricity + hardware costs), and worse quality. Better to outsource to providers who have optimized everything and costs are distributed/amortized across all users.
Perhaps someday local llms will be sufficient for coding etc, but it's going to be a while.
Comments
+1. i have similar workflow and i use local models on igpu so token cost is free and electricity cost is in cents.
I cannot imagine local models running on an igpu could get anything close to either a useful plan or execution of a plan. I've tested Qwen 3.5 27B locally and its solutions to coding problems are usually flawed and running in thinking mode is too slow and that's on a discrete GPU. How do you get anything useful done on a model that runs on an igpu?
yeah, i dont understand, whatsoever, the folks who are running models locally other than for privacy/curiosity. It is surely FAR slower, more expensive (electricity + hardware costs), and worse quality. Better to outsource to providers who have optimized everything and costs are distributed/amortized across all users.
Perhaps someday local llms will be sufficient for coding etc, but it's going to be a while.