I think people do not understand that the statement "you should always and only use encryption" is seriously flawed when it comes to protocols used by millions of applications.
If you are starting a new protocol for a wide audience you cannot just say "encryption is required". Which encryption protocol will you require? And will you stick with it forever? Even when it will be broken?
So you want a bit of forward compatibility and you add a version identifier to wire protocol the or some other form of negotiation. Well done, now you are opening your self to protocol downgrade attacks. You can try to fight this as much as you want, but will always be back to square one: support for non-updated applications: drop them or support them even thou you know there is a security problem in supporting them?
All the non-public SMTP servers I have used in the last years require the use of STARTTLS-enabled connections. So they are dropping old clients, but they can do that because they have a tight control over the "environment" (their students, their employers, etc). Others do not have this luxury.
You have it exactly backwards. If you are designing a new protocol, you do mandate encryption. Look at eg. recent new Web protocols, they all do it: WebRTC, WebSockets, SPDY.
Yes, the crypto engineering can be bewildering for non-experts, but it's just engineering at the end of the day.
In many cases there are defenses against downgrade attacks.
There is even hope for the web browsery scenario (have to be back compatible with SSL + connect to unknown/untrusted server, same as email): See http://tools.ietf.org/html/draft-bmoeller-tls-downgrade-scsv...
My idea is that mandating encryption and allowing version/cipher negotiation are more or less the same thing. They are not the same thing, but they lead to similar conclusions.
What should be the response of a client when a server tells it during the cipher negotiation that the only cipher that it (the server) can use are those considered insecure by client?
Should the client proceed for the sake of interoperability? Or should the client raise and error and say "sorry, no secure connection can be established"?
Whatever you answer to that question is, why doesn't it apply to STARTTLS As well? By not allowing STARTTLS, the server has negotiated a set of cipher (i.e. {NULL-ENC}) that the client considers insecure. What should the client do? Close the connection? (You can configure your program to do so right now.) Or continue because you really want to send that email? (This is, sadly, the current configuration of many applications.)
Back to the original point, I think that the important thing here is not only to embed secure layers in protocols, but also (and mainly) to convince the users that it is OK for an application to answer "Sorry, no secure connection is available. Yes, I could set up an insecure connection, but I will not do that."
What should be the response of a client when a server tells it during the cipher negotiation that the only cipher that it (the server) can use are those considered insecure by client?
Should the client proceed for the sake of interoperability? Or should the client raise and error and say "sorry, no secure connection can be established"?
These are solved problems though; it's not as though this is a new consideration. This has been a 'problem' since SSL's implementation. If you can't agree on a cipher spec, you don't connect.
The issue with your argument in particular is that you're basically saying 'we shouldn't require encryption because it's too complicated to make a few choices', which implies to me that you think not using encryption is a better choice (since any TLS implementation requires these issues). Your argument, then, is an argument against SSL/TLS encryption in any respect, as opposed to mandatory encryption.
Back to the original point, I think that the important thing here is not only to embed secure layers in protocols, but also (and mainly) to convince the users that it is OK for an application to answer "Sorry, no secure connection is available. Yes, I could set up an insecure connection, but I will not do that."
This is pretty common; in fact, most software will just say that it can't connect, with no more detailed explanation unless you dig into it. I've also had issues where the default configuration for postfix on Ubuntu (STARTLS enabled, self-signed certificate) is rejected by some clients, requiring me to disable authentication on those clients (which required a secure connection) and whitelist their IP addresses (insecure!).
Two rounds of negotiation means two points of failure. TLS already has a version/cipher negotiation step; it's built into the protocol, and your library code will contain an implementation, where it can be audited. Duplicating that functionality per-protocol is dumb.
I think people do understand your caveat, just that other concerns, such as providing actual security, override this. As the article argues, optional encryption does not work.
If you have authentication in the first version of your protocol, you can survive downgrade attacks by authenticating the version number.
For example, my email server will accept any email encrypted or not, but will only allow authentication over a channel if it's encrypted. But encryption is not optional for the latter case.
Email clients that attempt to authenticate won't send passwords over unencrypted plain-text connections, so there's no real vulnerability, unless a client is broken.
Sending the content of the email unencrypted is a pretty big vulnerability. Emails content is almost as sensitive as passwords and one would hope it would be systematically encrypted.
Comments
I think people do not understand that the statement "you should always and only use encryption" is seriously flawed when it comes to protocols used by millions of applications.
If you are starting a new protocol for a wide audience you cannot just say "encryption is required". Which encryption protocol will you require? And will you stick with it forever? Even when it will be broken?
So you want a bit of forward compatibility and you add a version identifier to wire protocol the or some other form of negotiation. Well done, now you are opening your self to protocol downgrade attacks. You can try to fight this as much as you want, but will always be back to square one: support for non-updated applications: drop them or support them even thou you know there is a security problem in supporting them?
All the non-public SMTP servers I have used in the last years require the use of STARTTLS-enabled connections. So they are dropping old clients, but they can do that because they have a tight control over the "environment" (their students, their employers, etc). Others do not have this luxury.
You have it exactly backwards. If you are designing a new protocol, you do mandate encryption. Look at eg. recent new Web protocols, they all do it: WebRTC, WebSockets, SPDY.
Yes, the crypto engineering can be bewildering for non-experts, but it's just engineering at the end of the day.
In many cases there are defenses against downgrade attacks. There is even hope for the web browsery scenario (have to be back compatible with SSL + connect to unknown/untrusted server, same as email): See http://tools.ietf.org/html/draft-bmoeller-tls-downgrade-scsv...
My idea is that mandating encryption and allowing version/cipher negotiation are more or less the same thing. They are not the same thing, but they lead to similar conclusions.
What should be the response of a client when a server tells it during the cipher negotiation that the only cipher that it (the server) can use are those considered insecure by client?
Should the client proceed for the sake of interoperability? Or should the client raise and error and say "sorry, no secure connection can be established"?
Whatever you answer to that question is, why doesn't it apply to STARTTLS As well? By not allowing STARTTLS, the server has negotiated a set of cipher (i.e. {NULL-ENC}) that the client considers insecure. What should the client do? Close the connection? (You can configure your program to do so right now.) Or continue because you really want to send that email? (This is, sadly, the current configuration of many applications.)
Back to the original point, I think that the important thing here is not only to embed secure layers in protocols, but also (and mainly) to convince the users that it is OK for an application to answer "Sorry, no secure connection is available. Yes, I could set up an insecure connection, but I will not do that."
What should be the response of a client when a server tells it during the cipher negotiation that the only cipher that it (the server) can use are those considered insecure by client?
Should the client proceed for the sake of interoperability? Or should the client raise and error and say "sorry, no secure connection can be established"?
These are solved problems though; it's not as though this is a new consideration. This has been a 'problem' since SSL's implementation. If you can't agree on a cipher spec, you don't connect.
The issue with your argument in particular is that you're basically saying 'we shouldn't require encryption because it's too complicated to make a few choices', which implies to me that you think not using encryption is a better choice (since any TLS implementation requires these issues). Your argument, then, is an argument against SSL/TLS encryption in any respect, as opposed to mandatory encryption.
Back to the original point, I think that the important thing here is not only to embed secure layers in protocols, but also (and mainly) to convince the users that it is OK for an application to answer "Sorry, no secure connection is available. Yes, I could set up an insecure connection, but I will not do that."
This is pretty common; in fact, most software will just say that it can't connect, with no more detailed explanation unless you dig into it. I've also had issues where the default configuration for postfix on Ubuntu (STARTLS enabled, self-signed certificate) is rejected by some clients, requiring me to disable authentication on those clients (which required a secure connection) and whitelist their IP addresses (insecure!).
Two rounds of negotiation means two points of failure. TLS already has a version/cipher negotiation step; it's built into the protocol, and your library code will contain an implementation, where it can be audited. Duplicating that functionality per-protocol is dumb.
IMAP and SMTP are not new, that's the point.
The context was "If you are starting a new protocol ..."
I think people do understand your caveat, just that other concerns, such as providing actual security, override this. As the article argues, optional encryption does not work.
If you have authentication in the first version of your protocol, you can survive downgrade attacks by authenticating the version number.
STARTTLS isn't just "optional encryption".
For example, my email server will accept any email encrypted or not, but will only allow authentication over a channel if it's encrypted. But encryption is not optional for the latter case.
Email clients that attempt to authenticate won't send passwords over unencrypted plain-text connections, so there's no real vulnerability, unless a client is broken.
Sending the content of the email unencrypted is a pretty big vulnerability. Emails content is almost as sensitive as passwords and one would hope it would be systematically encrypted.