What algorithm could/should be used to encrypt files for storage on untrusted servers?

Question

I need an encryption algorithm that works fast on large files and doesn't increase the file size. It should use a key to encrypt/decrypt data. The files will be sent using REST and over HTTPS, but a good encryption/decryption method with a key is mandatory (to be used in a website).

The files will be saved on a 3rd party servers, and although the files are sent through HTTPS, they must be stored encrypted on the remote servers and decrypted when retrieved. The key will not be distributed. The key will be stored on my trusted server, as it is also the receiver! I want to encrypt the file, put it on the 3rd party server, then when I need it I can retrieve it and decrypt it.

Answer 1

The question still lacks detail, so this answer will be equally vague and hand-waving.

Any modern cipher exceeds consumer disk/network IO speeds by a significant margin, you can pair any modern block cipher (AES is probably the best choice here) with a streaming mode of operation such that the file size is left intact, such as CTR or CFB. However, you still need integrity on top, so you also want a fast and strong hash function to use as a HMAC (or use AES in an authentication-capable mode, but such modes are not very widespread yet). A good hash function would be SHA256 (reasonably fast).

There is no "best" algorithm, there are only those that meet your needs and those that don't. Your needs are security and speed - choose accordingly. I gave some fairly generic suggestions above.

Of course, the better choice would still be to use something that's already production-ready, such as OpenSSL. This way you don't need to implement it yourself and just have to learn the API and make a few function calls at either end of your network sockets, and you don't have to worry about security problems (or at least, not as much).

Again, you should not be implementing cryptography if you don't know what you are doing, so your comments such as "I wish I could know that by my self" are worrying to say the least...

Answer 2

It seems your major requirement is that the file size not increase. This is possible, but at the sacrifice of some security (namely integrity and authentication), so it seems you will only be able to provide confidentiality. If the file size can increase slightly (say no more than 300 bits) you should encrypt with an authenticating mode such as GCM.

As Thomas points out, streaming modes will not alter the file size, so use one of those. There are some things you have to realize though if you go this route. Streaming modes are malleable. This means that an attacker (i.e., the 3rd party server storing the data) could make changes to the ciphertext which have a predictable change to the plaintext upon decryption.

In CFB, if the attacker flips one bit of the ciphertext, it will flip the corresponding bit of the plaintext, plus the next block will be completely scrambled. In OFB, if you flip one bit of the ciphertext, upon decryption, the corresponding plaintext bit will be flipped. Same with CTR, change one bit of ciphertext and you will flip the corresponding bit of plaintext.

This could have disasterous consequences. For example, if the attacker knows that the file is a text file of the form Transfer 500.00 dollars to mikeazo, the attacker (me in this case) could change that to say Transfer 50,000 dollars to mikeazo.

Another thing to keep in mind is the IV or Nonce required by streaming modes. These have their own requirements and will have to be stored or communicated some how. They can also be tampered with potentially, which results in problems (e.g., perhaps only one block of ciphertext will be unrecoverable, or maybe the entire thing will be unrecoverable).

A final aspect to consider is keying. How will the key be distributed?

My point is, yes, you can use a streaming mode with AES to encrypt a file and not increase the file size. But, there is a plethora of unintended consequences that you (or whoever is mandating this) is not considering. If this is to be used in a production system for a business, you are putting yourselves at great risk, and if that is the case, hire a professional. It will save you money in the long run. If that is not an option, use well established standards/software. I'd recommend something like GPG if the licensing works for the application.

Answer 3

For the problem as stated, a stream cipher may not work. Stream ciphers need to either have a unique IV (increasing file size) or a single-use key. Otherwise they leak the XOR of any two files encrypted with the same key and IV.

So, unless there's a side-channel that can be used for unique IVs (like file names that are known unique), AES in CTR or OFB mode would be insecure with key reuse. Instead, I would use either CFB or CBC, both of which offer at least some security even when IVs collide.

The first blocks will still leak information, slightly less so with CBC (which may be vulnerable to other attacks). Any unauthenticated encryption also allows an attacker to alter the files. So my real suggestion is to accept the extra space you lose to a strong IV and authentication tag, and use either AES GCM or AES CTR + HMAC.