Skip to content

Comment on How Not to Encrypt a File – Courtesy of Microsoftparent

Comments

I myself don't really understand IVs

Time to drop some knowledge!

IVs are used in a number of places in cryptography, so I'll just pick one (easy) example.

Consider the stream cipher ChaCha20. You can think of ChaCha20 as a black box. You input a key and an IV and out you get a really, really long stream of uniformly random bytes. (This is a simplification but sufficient here). ChaCha20 works in such a way that having any or all of the output stream doesn't help you figure out what the inputs were. It's irreversible. ChaCha20 is also deterministic; the same input will give the same output.

You can then use the output of random bytes to encrypt a message by XORing with your plaintext. To later decrypt, you feed the same key and IV, get the same stream, XOR the ciphertext with it, and by the property of XOR you'll get the plaintext.

Now why is there an IV? Let's consider a ChaCha without an IV. The system works like so:

    R = ChaCha(Key)
    Ciphertext = Message ^ R
So let's encrypt two different messages:
    R = ChaCha(Key)
    Ciphertext1 = Message1 ^ R
    R = ChaCha(Key)
    Ciphertext2 = Message2 ^ R
Notice how R is the same for both messages? Again, ChaCha is deterministic; the output is the same for the same inputs. Since the key is the same, R is the same. Now an attacker, knowing this, can do this
    Q = Ciphertext1 ^ Ciphertext2
What does Q end up being? Let's look:
    Ciphertext1 ^ Ciphertext2
    = Message1 ^ R ^ Message2 ^ R
    = Message1 ^ Message2
So Q ends up being equal to the XOR of the two messages. That's really bad. The xor of two messages might be enough to tell the attacker what the messages are, especially if the messages are predictable (like english text). But maybe that's not scary enough. Well there's another attack. What if you're encrypting a data format with a header. Headers often have the same data in the same places. So the attacker knows part of the message. Uh oh...
    R = Ciphertext1 ^ Message1
If the attacker knows the message (or any parts of it) they can recover the R of those parts. And now, since your key is always the same and your R is always the same, all the other messages you encrypt will have those bytes exposed.

This is where IVs come in:

    R = ChaCha(Key, IV)
IV should be unique per message. That means that every R is different! None of the above attackers work anymore. XORing two ciphertexts together returns gibberish:
    R1 = ChaCha(Key, IV1)
    Ciphertext1 = Message1 ^ R1
    R2 = ChaCha(Key, IV2)
    Ciphertext2 = Message2 ^ R2

    Ciphertext1 ^ Ciphertext2
    = Message1 ^ R1 ^ Message2 ^ R2
And if the attacker knows the message, all they can recover is R1 or R2 (or any R). But that's useless, because since all your IVs are unique that R will never be seen ever again.

That's the point of IVs.

why the IV isn't required to be able to decrypt the file again.

It is required. Obviously you need all the inputs to ChaCha to get the byte stream again, to decrypt the message.

Now sometimes the IV is known from the protocol. So say you're using ChaCha to encrypt network traffic. You might set the IV equal to the packet number. So both sides already know the packet number.

But you always need the IV to decrypt.

and from my perspective I'm left with no clearer of anuidea about why IVs shouldn't be considered secret,

Consider again ChaCha20 as a blackbox. Key+IV goes in, stream of bytes comes out. There's no way to reverse that without the key (and IV). Since the attacker doesn't know the Key, they can't reverse it. Knowing the IV doesn't help.

Another way to think about it is that, instead of accepting a 256-bit key and a 64-bit IV, it's really just a 320-bit key. Knowing 64-bits of a 320-bit key doesn't help break a cipher. The cipher is still 256-bits strong. So you can share the IV without affecting security.

BIG NOTE: It's important that an IV is always unique. If an IV is ever re-used, the above attacks become available again because R will be the same for two messages.

Hope that helps. This is only one way that IVs are used. In ChaCha20 it's called a nonce, because ChaCha20 is geared towards usage on network protocols where the above trick of using packet number is applied. For block ciphers there are various cipher modes that get used, and most of them need an IV. The purpose is always the same; to make this "session" of encryption unique.

There's another way to use IVs, and I think they re-affirm the concept of what an IV actually is. Let's say you have a cipher that only accepts a key! No IV (like AES). You still want to make your encryption sessions unique. A way to do that is this:

    TempKey = HMAC (IV, Key)
And then use TempKey. HMAC is a form of hash. In this case it lets us combine a Key and IV in an irreversible way, yielding a new key. TempKey will be the right size key for the cipher (say, 256-bits). What this is doing is giving us a unique key for every encryption session. And that's the heart of IVs. And in many ways, ChaCha20 is doing exactly that. It's hashing together Key and IV and using the output hash to generate a long stream of random data that can't be reversed back to the key+IV.

(and in case you're wonder, yes, you can use a cryptographically secure hash function alone to build a stream cipher like ChaCha. It'll just be _really_ _really_ slow, because hash functions are really, really slow compared to ChaCha.)

Thanks for spending the time explaining this.

Happy to.

Is this something people find interesting? I was thinking of doing a small guide/tutorial/course where I teach these basics of cryptography, while building up to a working file encryption tool written completely from scratch. Probably such a thing exists already, but /shrugs these kinds of questions always seem to come up.

This is risky. Unless you are a pro, it is generally not a great idea to publish a "how-to" for crypto because of the risk you might get it wrong in subtle ways that now propagate through the ecosystem.

I know all the subtle things.

That said, I wouldn't write the course for people intending to become cryptographers or cryptographic engineers. That would require a university grade program. It would be geared towards people who have a curiosity of the inner machinations of encryption. Ya know, like the people who come to Hacker News, read articles like this, and ask questions.

Note that AES uses IVs in CBC mode. It is incorrect to say that AES does not use IVs.

CBC mode is not specific to AES.

EDIT: To elaborate. AES does _not_ use IVs. It has no support for them whatsoever. It takes as input only a key and plaintext/ciphertext. This is in contrast to other block ciphers like Threefish which do.

AES, in most applications, has to be used in constructions that require IVs. But that's distinctly different. IVs are bolted onto AES.

"It is incorrect to say that AES does not use IVs." is patently false. It'd be like saying "It is incorrect to say that AES does not use tweaks."

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.