I've been thinking about this problem with law a bit lately. It seems like our laywers & politicians want every legal question to be completely decidable, and this attitude leaks into popular culture too (or is it the other way around?), hence this article's preoccupation with the fact that good or evil intent cannot (as far as we know) be identified by any mechanical process.
This type of thing is of course a problem in computer science, but the law is not a computer, and I wish we could stop treating it like one. With humans involved, it's quite feasible to make decisions about undecidable things. This process is called exercising judgement, and we hire judges to do it. Frankly, I don't see what the problem is with a "don't be evil" clause. Pretty much everyone understands what it means in most any given circumstance, even if we can't write down a decision procedure for it.
I understand that part of the motivation here is the urge to CYA and offload all accountability for bad decisions from people to "the law". Not sure what to do about that.
(I wonder, can we make a proof of the undecidability of the good-vs-evil problem? I don't think it'll be as easy as for the halting problem; we know how to make a program halt or not halt, but it's unclear how to make a program "do something evil" or "do something good".)
There is a very good reason we want legal questions to be completely decidable - if they're not, people can't be sure ahead of time if they're breaking the law, which makes it ethically very problematic to actually enforce that law. ("You broke the law, and we're going to fine you." "But it was impossible to know that it was against the law!" "...")
The law should be predictable. This is obviously also not perfectly attainable, but we should not deliberately put ambiguities in there. Unpredictable laws directly lead to rule by men (as opposed to rule by law) and there's a lot of history and tradition on why that is undesirable.
Comments
I've been thinking about this problem with law a bit lately. It seems like our laywers & politicians want every legal question to be completely decidable, and this attitude leaks into popular culture too (or is it the other way around?), hence this article's preoccupation with the fact that good or evil intent cannot (as far as we know) be identified by any mechanical process.
This type of thing is of course a problem in computer science, but the law is not a computer, and I wish we could stop treating it like one. With humans involved, it's quite feasible to make decisions about undecidable things. This process is called exercising judgement, and we hire judges to do it. Frankly, I don't see what the problem is with a "don't be evil" clause. Pretty much everyone understands what it means in most any given circumstance, even if we can't write down a decision procedure for it.
I understand that part of the motivation here is the urge to CYA and offload all accountability for bad decisions from people to "the law". Not sure what to do about that.
Further reading: http://www.newyorker.com/arts/critics/atlarge/2012/01/30/120... ; appeared on HN yesterday; from paragraph 7, argues that these attitudes are partly responsible for the current incarceration problem in America.
(I wonder, can we make a proof of the undecidability of the good-vs-evil problem? I don't think it'll be as easy as for the halting problem; we know how to make a program halt or not halt, but it's unclear how to make a program "do something evil" or "do something good".)
There is a very good reason we want legal questions to be completely decidable - if they're not, people can't be sure ahead of time if they're breaking the law, which makes it ethically very problematic to actually enforce that law. ("You broke the law, and we're going to fine you." "But it was impossible to know that it was against the law!" "...")
The law should be predictable. This is obviously also not perfectly attainable, but we should not deliberately put ambiguities in there. Unpredictable laws directly lead to rule by men (as opposed to rule by law) and there's a lot of history and tradition on why that is undesirable.
Very good point. Thank you.