1
$\begingroup$

I'm trying to decide on which ciphermode to use for a programming excercise. I have a file that I need to encrypt without saving unencrypted data to storage and without storing the entire file in memory.

I've opted for streaming the file, in a fixed size into a buffer, encrypting said buffer, write said buffer to storage and clear the buffer before doing it all again with the next slice of data.

Once the encrypted chunks had been written to storage, they could be consolidated into a single file, which would have to be split into chunks again for decryption.

The above seems like a standard way of doing it, from what I could gather, but I'm not entirely sure as to which ciphermode to use.

I was wondering which of AES-GCM and AES-CBC would be best to use?

Ideally I'd like to choose a solution that is "encrypt-then-mac", but I'm unsure which of those fit the bill here?

The programming language I'm doing this in is Golang, if that matters to anyone.

$\endgroup$
  • $\begingroup$ If you're not sure whether you need authentication, the thing you should reach for is authenticated encryption like AES-GCM; forget that ‘modes of operation’ or AES-CBC exist. But also consider using something that was defined for a streaming API: github.com/miscreant/miscreant That said, make sure that the legitimate semantics of your system is defined even if the stream is truncated or missing pieces, because the attacker gets to control which pieces get dropped. $\endgroup$ – Squeamish Ossifrage Feb 8 '18 at 16:23
5
$\begingroup$

I was wondering which of AES-GCM and AES-CBC would be best to use?

If these two are your option, then it is essentially a no-brainer. Always use AES-GCM unless you really know what you are doing.

Ideally I'd like to choose a solution that is "encrypt-then-mac", but I'm unsure which of those fit the bill here?

That's what AES-GCM does internally. It encrypts the data with AES-CTR and then uses a fast, specialized MAC to authenticate the ciphertext.

Once the encrypted chunks had been written to storage, they could be consolidated into a single file, which would have to be split into chunks again for decryption.

One thing to look out for when doing streamed decryption is that the resulting plaintext must not be used until the decryption has finished. The reason being that until you have processed everything (including the authentication tag at the end), you can't be sure that the data hasn't been tampered with. There are workarounds to this, that allow you to chunk the data and authenticate it individually using incrementing nonces to prevent re-ordering attacks, but even then there may still be problems with substitution attacks (ie where old versions of chunks are used to substitute current chunks).

$\endgroup$
  • $\begingroup$ Those are the 2 available to me (From what I know of), unless I go around using something outside the standard library of golang. AEs-GCM was my first choice, as I had an idea that it used encrypt-then-mac, but I starting wondering about the fact that using the same nonce/iv isn't recommended. Which would mean that each chunk would need to have a separate nonce, which could be a nightmare to manage with a large file. Or am I wrong here, is it acceptable to use the same nonce for the chunks? I also read somewhere that AES-GCM has trouble encrypting files larger than 64Gb? $\endgroup$ – MadsRC Feb 6 '18 at 20:20
  • $\begingroup$ @MadsRC yes, encrypting two different messages under the same nonce, is fatal for AES-GCM (much like with CTR). And yes, you really should not encrypt more than 64GB per nonce-key pair (that is, switching the key or the nonce will give you another 64GB). $\endgroup$ – SEJPM Feb 6 '18 at 20:35
  • 2
    $\begingroup$ Note using a predictable IV for CBC is almost as bad as using a duplicated one for CTR or GCM -- and avoiding predictability is sometimes harder than avoiding duplication. $\endgroup$ – dave_thompson_085 Feb 7 '18 at 4:15
1
$\begingroup$

I've opted for streaming the file, in a fixed size into a buffer, encrypting said buffer, write said buffer to storage and clear the buffer before doing it all again with the next slice of data.

For random access files streaming is an option. But you could also memory map the file; that probably requires less copying and may allow for parallel decryption. Very often the IO speed will be less than the encryption/decryption speed, so it depends if this will help all that much.

You could also use this for in place encryption / decryption but remember that you may get into trouble if the operation is stopped in the middle, leaving you with a half-encrypted file.

Once the encrypted chunks had been written to storage, they could be consolidated into a single file, which would have to be split into chunks again for decryption.

As long as the decryption procedure is compatible with the encryption procedure everything is fine, yes. In principle you could deviate from the encryption procedure but in general you should probably keep it as symmetric as possible.

I was wondering which of AES-GCM and AES-CBC would be best to use?

CBC can do if you just require confidentiality for in-place encryption. Most of the time just requiring just confidentiality isn't enough. If there is a possibility of automated decryption then padding oracles could also apply; in that case confidentiality is lost as well. Using GCM is recommended.

The programming language I'm doing this in is Golang, if that matters to anyone.

Not for defining the protocol, but it could matter for implementing it in a secure fashion. Golang has a good set of cryptographic API's so it should be usable for this kind of functionality - I haven't reviewed it myself though.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.