Deciding on an AES cipher mode for chunked data

Question

I'm trying to decide on which ciphermode to use for a programming excercise. I have a file that I need to encrypt without saving unencrypted data to storage and without storing the entire file in memory.

I've opted for streaming the file, in a fixed size into a buffer, encrypting said buffer, write said buffer to storage and clear the buffer before doing it all again with the next slice of data.

Once the encrypted chunks had been written to storage, they could be consolidated into a single file, which would have to be split into chunks again for decryption.

The above seems like a standard way of doing it, from what I could gather, but I'm not entirely sure as to which ciphermode to use.

I was wondering which of AES-GCM and AES-CBC would be best to use?

Ideally I'd like to choose a solution that is "encrypt-then-mac", but I'm unsure which of those fit the bill here?

The programming language I'm doing this in is Golang, if that matters to anyone.

Answer 1

I was wondering which of AES-GCM and AES-CBC would be best to use?

If these two are your option, then it is essentially a no-brainer. Always use AES-GCM unless you really know what you are doing.

Ideally I'd like to choose a solution that is "encrypt-then-mac", but I'm unsure which of those fit the bill here?

That's what AES-GCM does internally. It encrypts the data with AES-CTR and then uses a fast, specialized MAC to authenticate the ciphertext.

Once the encrypted chunks had been written to storage, they could be consolidated into a single file, which would have to be split into chunks again for decryption.

One thing to look out for when doing streamed decryption is that the resulting plaintext must not be used until the decryption has finished. The reason being that until you have processed everything (including the authentication tag at the end), you can't be sure that the data hasn't been tampered with. There are workarounds to this, that allow you to chunk the data and authenticate it individually using incrementing nonces to prevent re-ordering attacks, but even then there may still be problems with substitution attacks (ie where old versions of chunks are used to substitute current chunks).

Answer 2

I've opted for streaming the file, in a fixed size into a buffer, encrypting said buffer, write said buffer to storage and clear the buffer before doing it all again with the next slice of data.

For random access files streaming is an option. But you could also memory map the file; that probably requires less copying and may allow for parallel decryption. Very often the IO speed will be less than the encryption/decryption speed, so it depends if this will help all that much.

You could also use this for in place encryption / decryption but remember that you may get into trouble if the operation is stopped in the middle, leaving you with a half-encrypted file.

Once the encrypted chunks had been written to storage, they could be consolidated into a single file, which would have to be split into chunks again for decryption.

As long as the decryption procedure is compatible with the encryption procedure everything is fine, yes. In principle you could deviate from the encryption procedure but in general you should probably keep it as symmetric as possible.

I was wondering which of AES-GCM and AES-CBC would be best to use?

CBC can do if you just require confidentiality for in-place encryption. Most of the time just requiring just confidentiality isn't enough. If there is a possibility of automated decryption then padding oracles could also apply; in that case confidentiality is lost as well. Using GCM is recommended.

The programming language I'm doing this in is Golang, if that matters to anyone.

Not for defining the protocol, but it could matter for implementing it in a secure fashion. Golang has a good set of cryptographic API's so it should be usable for this kind of functionality - I haven't reviewed it myself though.