How to Screw Up Encryption

Today, we mess up encryption application in two primary ways. We either try to create our own (always a bad idea) or misconfigure protocols to use weak algorithms (also a bad idea, but it's surprisingly easy to do).

With respect to the first way we screw up encryption, we need to be a bit more detailed with respect to what we mean. When I say "create our own", I don't mean "try to create our own encryption algorithm". I mean, nobody's going to do this. Not that nobody can, after all the initials in RSA stand for real folks, but that it's likely that you don't know anybody who can do this. And that you don't know anybody who knows anybody who can do this. And for the most part we all realize this, so we don't try. Go us!

What I mean is "create our own encryption protocols". This kind of thing comes up more frequently than you'd think. When we're doing this kind of thing, we're using established encryption algorithms in some kind of a sequence to protect information. Examples include things like network communication, protecting data at rest over time, that kind of thing. And usually, you need to provide not only confidentiality, but integrity and some kind of authentication too. To do this, you'll use established protocols, things like AES and SHA and ECC. These are all strong algorithms (by which I mean provably strong), but by making small mistakes here and there, you can make them much, much weaker. Things like not using truly random initialization vectors, or not salting, or reusing keys. Or providing automatic fallbacks to known compromised algorithms. Or mis-designing your protocol so that attackers can circumvent encryption entirely via out-of-sequence messages.

Those last two popped up in OpenSSL and LibSSL, respectively, so even folks with lots of experience in these areas screw it up. If they do, imagine the kind of damage folks that don't work with crypto all day can do.

The second way we mess up encryption is misconfiguration. Really, this is pretty closely related to the kinds of mistakes you can make designing your own protocol. Usually, when you design your own protocol you don't put as many options in as other established protocols do, so you hard-code your mistakes. When configuring established protocols, you can make the same kinds of mistakes. Specific things to look out for are allowing the implementation to fall back to old, insecure protocols (like SSL3.0) and reusing keying material.

Trust me, whenever you're implementing encryption between systems, always try to use established systems with significant community or commercial support. Can you use an established system from a vendor or open source project? Do it. And the more documentation the better. Projects like OpenSSL or the Noise Framework are excellent options, and between those two you will probably take care of all your crypto needs. And make sure you know where you can get help and configuration information before you start. You don't want to be panicking over some small detail when you're hours from an implementation deadline. And make sure you give yourself enough time! Don't rush. This stuff is complicated.

If you avoid these mistakes and take my advice, you shouldn't have any problems implementing crypto systems the next time around. Good luck in advance!