1
$\begingroup$

I'm mainly trying to verify large file chunks which are encrypted each alone with a different key/salt/nonce for each chunk then appended to the whole file in the end to form 1 file.

One thing I wanted to implement was the principle of not outputting plain data of any chunk to the user unless every chunk of the file authenticates successfully. If 1 fails, the whole operation fails.

To do this, if the file has multiple chunks, I do 1 pass to authenticate all chunks, then another pass to decrypt them if the first pass authenticates.

I'm using the library provided by .net to do this and it seems to output the plain chunk to a span if that chunk authenticates which would technically put the decrypted chunk in memory, technically allowing someone to see what's happening in the memory for the time it is there. How am I supposed to ONLY authenticate without doing any output whatsoever to any destination, and can that library do it somehow (i.e. am I missing anything?) or should I use another library that provides this functionality?

It seems very odd to me that this is the behaviour, compare it to a group of people standing outside your door, you ask their ID one by one but it doesn't mean that will let them enter automatically if they verify successfully.. maybe you want the full group in at once..

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ If an adversary is able to observe intermediary results such as deciphered plaintext, and unless the decryption is performed in some secure environment (like a Smart Card or HSM), it's to fear that by the same means they can also observe the key (or round keys, which allows to walk back at the key) even if the output of ciphertext is suppressed before reaching memory. Thus I think such suppression effort is futile. I second this answer recommending to worry for overall file integrity facing reshuffle of file segments, truncation... $\endgroup$
    – fgrieu ♦
    Aug 1, 2022 at 8:44
  • 1
    $\begingroup$ @fgrieu I am protected against reshuffle of file segments so I guess I will not worry about authentication without decryption then. $\endgroup$
    – Elie-M
    Aug 1, 2022 at 11:30

1 Answer 1

Reset to default
1
$\begingroup$

I'm mainly trying to verify large file chunks which are encrypted each alone with a different key/salt/nonce for each chunk then appended to the whole file in the end to form 1 file.

This sounds very inefficient. In fact, it sounds like what you're doing may not be linking the chunks together at all, which would be a vulnerability.

You need to ensure that:

  1. The file cannot be truncated.
  2. None of the chunks can be reordered or duplicated.

The first can be accomplished by including the length of the entire file in the associated data of the first chunk. Alternatively, you can use something like the STREAM construction, but that's slightly more complicated. It's used by projects like age.

The second can be done using a counter nonce or by including the previous authentication tag in the associated data of the next chunk.

I would recommend incrementing the nonce for a given key and using a unique key for each file. The initial nonce can be 0 or random.

One thing I wanted to implement was the principle of not outputting plain data of any chunk to the user unless every chunk of the file authenticates successfully. If 1 fails, the whole operation fails.

You should either:

  1. Use HMAC-SHA256 or HMAC-SHA512 over the entire file (if you want to stick with the .NET library). BLAKE2b or BLAKE3 would be faster. However, this really defeats the entire point of using AES-GCM in my opinion, and you'd be better off just doing Encrypt-then-MAC for each chunk, which would offer committing security.
  2. Don't authenticate the entire file before decrypting the first chunk. Instead, you simply throw an error and erase all of the output when verification for a chunk fails.

I would recommend 2, regardless of whether you want to use AES-GCM or Encrypt-then-MAC.

Improve this answer
$\endgroup$
6
  • $\begingroup$ I already have taken care of the chunks being reordered or duplicated this is besides my point and the chunks are linked 100% sure. The main point here is if there was a way to authenticate without doing any decryption operation. I was doing as you suggested and deleting the output but you mentioned my way of doing full authentication first is inefficient, but the whole operation is extremely fast even on large files. And I don't see any issue with efficiency if the operation clocks at acceptable times by us, which it is and far from the upper limits. $\endgroup$
    – Elie-M
    Aug 1, 2022 at 11:27
  • $\begingroup$ I just wanted to make sure, and it sounds like a potentially overcomplicated way of doing things. What you're doing involves processing everything twice when you're using an algorithm designed to process everything in 1.5 passes is my point. Using GCM for that is not standard, and BLAKE3 on large inputs should be faster than GCM. Thus, I don't think there's reasonable justification to use GCM for this purpose. $\endgroup$
    – samuel-lucas6
    Aug 1, 2022 at 13:37
  • $\begingroup$ My reason to use AES-GCM is simple: I like it. Might sound dumb, but I was using asymmetric encryption and I wasn't really liking it and I saw someone mention AES-GCM for authenticated encryption/decryption and I gave it a try and I loved everything about it, and it was able to cater to all my needs and was able to implement it easily and cleanly, while keeping integrity and speed and all that. But since I was encrypting each chunk with its own random key and salt (using DEK/KEK model both with Argon2) I wanted to authenticate the whole chain before writing plain text, as someone recommended. $\endgroup$
    – Elie-M
    Aug 1, 2022 at 13:50
  • 1
    $\begingroup$ Yep, that's why AEADs are so popular. I was the person who recommended KEK/DEK to you, but I didn't mean a unique key and salt for each chunk, just per file. That's a lot simpler without any security concern still because you'll be changing the nonce for each chunk. I understand, but it's not necessary with chunked encryption. $\endgroup$
    – samuel-lucas6
    Aug 1, 2022 at 14:04
  • $\begingroup$ But since the decryption works based off of the ciphertext and the nonce, I would have to load the whole file in memory just to authenticate the tag which is something I don't want to do, so instead I let each chunk have its random key and nonce and authenticate itself, and let the associated data authenticate the full chain, that way I only load a chunk in memory not the full file, especially when you have large files like 10 GB+, also especially that C# has a 2 GB limit for byte arrays. For reference my chunks are 256 MB to keep a good performance. Files under 256 go at once. $\endgroup$
    – Elie-M
    Aug 1, 2022 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.