The difference is that a car is known and expected to work reliably. An LLM is known and expected to randomly do crazy shit because that is how the machine works. Therefore, if you are using it as if it were reliable, you are being negligent and should be held accountable.
Companies don't market their chatbots that way though. If they mislead someone into thinking their product is anything other than a completely unreliable chat bot and a person falls for their lies the company should share the responsibility. If a company who makes a chatbot advertises that it can be used for something and a consumer uses it as directed and it causes harm the company should be responsible.
Comments
The difference is that a car is known and expected to work reliably. An LLM is known and expected to randomly do crazy shit because that is how the machine works. Therefore, if you are using it as if it were reliable, you are being negligent and should be held accountable.
Companies don't market their chatbots that way though. If they mislead someone into thinking their product is anything other than a completely unreliable chat bot and a person falls for their lies the company should share the responsibility. If a company who makes a chatbot advertises that it can be used for something and a consumer uses it as directed and it causes harm the company should be responsible.