I'm not sure why there's a problem before even trying.
Say for instance you have a program that keeps recipes. In your configuration file, there's settings such as metric/imperical units, allergies, diets, type of stove in your kitchen, and some theming (font, size, color...). You put them all in a file that you can parse, but aunt tillie messes up the formatting and the program breaks.
Instead of changing the config by hand, an LLM could supply the diff according to instructions in English. I really don't see why this would be "Terrible for prod". If the LLM screws up, you're simply back to square 1 and aunt tillie will call you just like she would before she had an LLM to fix her computer.
Same model. Same hardware... Depends. If you sacrifice speed then no. Ieee754 is pretty specifically specified, the issue is that it's not associative. If you get the associativity correct, then there's no issue. Associativity usually dies due to scheduling
Comments
Thats the problem isnt it? LLMs arent deterministic. Terrible for prod
I'm not sure why there's a problem before even trying.
Say for instance you have a program that keeps recipes. In your configuration file, there's settings such as metric/imperical units, allergies, diets, type of stove in your kitchen, and some theming (font, size, color...). You put them all in a file that you can parse, but aunt tillie messes up the formatting and the program breaks.
Instead of changing the config by hand, an LLM could supply the diff according to instructions in English. I really don't see why this would be "Terrible for prod". If the LLM screws up, you're simply back to square 1 and aunt tillie will call you just like she would before she had an LLM to fix her computer.
Llms are absolutely deterministic if you want them to be.
Still a terrible idea for config
Don't you need the same model, on the same hardware, with the same prompt, and temperature set to 0 to make it deterministic?
Same model. Same hardware... Depends. If you sacrifice speed then no. Ieee754 is pretty specifically specified, the issue is that it's not associative. If you get the associativity correct, then there's no issue. Associativity usually dies due to scheduling