But you cannot just copy/paste the code from LLMs and push to production

For how long though? this is the state of LLMs today, but I'm not sure this will be the case in 3-5 years.