3
$\begingroup$

I've been looking for a fast, powerful and robust algorithm for encrypting large files with authentication . I would like to implement the algorithm on my own without using third party software. I've used different modes of AES but the software implementation is prone to side channel attacks. Besides the most commonly used AES 256 GCM has an encryption with authentication size limit of 64 GB. I've tried other famous ciphers like Xchacha20/ XSalsa20 with poly1305, but I cant get around their respective size limits (I can only encrypt upto 256 GB with XChaCha20 - Poly1305 - IETF version ) !

  • Is there a powerful enough cipher other than the above that I can use for large scale encryption with authentication ?

  • What modifications should I apply to existing algorithms like Xchacha20 / XSalsa20 with Poly1305 for unlimited encryption with authentication?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ For chacha20 you can use the nonce area as a counter (so you would end up with a maximum of 2^131 byte filesize), just make sure to change the key for each file. If you want to use a nonce as well then go for XChacha20 and modify it to only use 128-bits for nonce and another 128-bits for counter (rather than 192 bits for nonce and 64 bits for counter). That being said, are you sure that the 2^67 bytes (- 64 if you use poly1305 as well) that the original chacha offers you is not enough for you? $\endgroup$
    – Bob Semple
    Commented Mar 26, 2020 at 14:14
  • $\begingroup$ @Bob Semple: Thankyou, for your comment ! I was trying to custom encrypt all the files in a 1TB harddrive (I am not specifically referring to disk encryption) ! $\endgroup$
    – Aravind A
    Commented Mar 26, 2020 at 14:19
  • $\begingroup$ In that case you might be interested in AES using the XEX or XTS mode. I think the limit is 2^64 bytes per key for it but I am not sure. $\endgroup$
    – Bob Semple
    Commented Mar 26, 2020 at 15:41
  • $\begingroup$ @Bob Semple AES XTS and XES have a lot of implementation difficulties if we implement it in software. I recently found Adiantum by Google that uses XChaCha20- Poly1305 for disk cryptography. I would like to know of AES alternatives... $\endgroup$
    – Aravind A
    Commented Mar 26, 2020 at 16:25
  • $\begingroup$ Adiantum also uses AES. AES is not that bad to implement in software, but we have AES-NI regardless. Anyway you could use pretty much any modern block cipher in the XEX mode, such as Serpent. $\endgroup$
    – Bob Semple
    Commented Mar 26, 2020 at 17:23

1 Answer 1

Reset to default
6
$\begingroup$

You shouldn't encrypt large files as if they were just one message; you should split them up into small chunks and encrypt each chunk separately with an AEAD, using some construction that:

  • Protects against modification, reordering, insertion and deletion of chunks;
  • Protects against truncation of the file;
  • Rotates keys once too much data has been processed by a single key.

One reason is that cryptographic libraries typically are implemented so that you need an entire message in memory to encrypt it. Another reason is that users of tools often expect that the output will be produced incrementally (so they can pipe it to other tools), but typical AEAD ciphers demand that you never output the decryptions of forged ciphertexts, so encrypting the whole file as just one message requires you to authenticate the whole thing before you produce any output. (Not that outputting authentic prefixes of a forgery is always safe.)

See:

Once you have adopted this paradigm, the message length and ciphertext size limits in ciphers that you're concerned about don't matter anymore, because each you only ever encrypt short messages and you can build in key rotation if you go past the ciphertext limit for a single key. For example the Libsodium secretstream API I link to above offers this feature:

  • Ratcheting: at any point in the stream, it is possible to "forget" the key used to encrypt the previous messages, and switch to a new key.

So if you just keep a count of how much data you've processed and instruct the library to rekey when it approaches a limit, you have, as the documentation puts it, "no practical limits to the total length of the stream."

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.