I think writing little plugins and drivers to do the thing you want with an LLM is something that could be built into a lot of software.

Containing the craziness within low level drivers sounds like the perfect way to do several great things: fuzz the OS for vulnerabilities, finally relieve embedded engineers of what they clearly hate so much, save good tech from the scrap heap. So many wins!