-1
$\begingroup$

I am implementing a cross platform AES cipher. Everything works fine except for super large file, those I encrypt by for example taking first 1kb of the file generating a cipher message (all chunks have same IV and password) then appending that part of the ciphered message to a new file. So to decrypt it we simply load in each 1kb at the time and do the opposite. And that works great up to a certain point.

When I append each ciphered chunk/part of the message I also append at the end of each chunk "@@@" for me to easily when decrypting knowing exactly where one chunk ends or is located.

Now it works 95% of the time, but luckily I found a bug, in case the AES cipher by randomness produces the (end of message symbol which was = "@@@") then decryption derails.

So is there possibly any kind of symbol or byte that AES-256 would NEVER produce?? in that case it would be fantastic to use that as the end of chunk symbol instead of my "@@@".

On a side note: I have tried using base64 also on the bytes and that works wonderful by just adding one "@" at the end of every chunk since base64 never outputs a "@" character. But base 64 just seems to make the file 10% bigger so if I can just avoid base64 and use bytes it would be optimal.

edit: It's cbc mode and looks like this

{iv}"@@@"{chunk1}"@@@"{chunk2}"@@@"{chunk3}......

With the base64 variant that works without any problems:

{Base64(iv)}"@"{base64(chunk1)}"@"{base64(chunk2)}.....
$\endgroup$
10
  • 3
    $\begingroup$ Why do you split your data manually? AES will split it anyway again, since it has a smaller block size. Also, it's probably not ideal to reuse the IV, depending on the mode of operation you just broke the security. Regarding the question: No, AES (for a fixed key) is a permutation, so it has to be able to output the full range. $\endgroup$
    – tylo
    Commented Dec 26, 2019 at 11:05
  • $\begingroup$ The issue is that no program/computer can load in a file of say 10gb at once and cipher it, We have to work in chunks that the computers memory can handle :( $\endgroup$
    – CoffeDev
    Commented Dec 26, 2019 at 11:08
  • $\begingroup$ awesome thanks for the confirmation, then i guess base64 encoding is the way to go? $\endgroup$
    – CoffeDev
    Commented Dec 26, 2019 at 11:10
  • 2
    $\begingroup$ If two chunks use the same key and IV an adversary can tell if they start with the same data and how long that data is (in increments of 16 byte, rounded down). $\endgroup$
    – SEJPM
    Commented Dec 26, 2019 at 12:56
  • 2
    $\begingroup$ Seeing the length of a shared prefix can be tolerable, if you deem it so, but it certainly isn't the strongest security notion CBC could achieve here (which would be that nothing is leaked about the chunks or the relations of the contents). But for example if an adversary could choose say the plaintext of the first chunk they could make guesses for the start of all others. $\endgroup$
    – SEJPM
    Commented Dec 26, 2019 at 13:04

2 Answers 2

4
$\begingroup$

Modern block ciphers encrypt a fixed-size chunk of plaintext as a same-sized chunk of ciphertext. Since the blocks are the same size and any byte sequence is a valid input, it's clear that every possible output sequence will be hit. Since encrypting and decrypting are inverse operations, it's easy to find an input that encrypts to any particular output - consider Encrypt(K, Decrypt(K, "@@@@@@@@")).

$\endgroup$
-1
$\begingroup$

Actually i solved this in a completely different way incase anyone is wondering. Instead of trying to split the bytes into a splittable list there was a more strict way to solve this by following a pattern. When encypting, each chunk would be a multiple of 16, and minus 1. So if you wanted each chunk to be 16 bytes then we would make each chunk 16-1=15bytes, and padding would make it into 16bytes after encryption.

So for decryption we can now simply say that each 16bytes in the file is one complete aes cipher chunk. And for the last chunk in the file it doesn’t matter how long it is since the chunk will be padded into 16bytes also :)

The only downfall to this method is that we need to know exactly what chunksize was used to be able to decrypt it. but thats okay for private encryption use.

$\endgroup$
9
  • 3
    $\begingroup$ That's still a horrible solution. Most languages have an update / doFinal function or even better, streaming functionality for CBC, which allows you to implement a simple read / write loop. All this mucking about is entirely unnecessary and potentially insecure and non-performant. $\endgroup$
    – Maarten Bodewes
    Commented Dec 28, 2019 at 11:19
  • $\begingroup$ @Maarten-reinstateMonica Oh wow, i thought most people here thought that this was a safe implementation with Authentication + IV + Ciphertext? Actually i would LOVE if there was an "update" function similar to how we hash very big files and do hash.update(..Extra bytes...). But in my case im trying to make a crossplatform encryption that works everywhere and the update function is nowhere to be found for AES-Rijndael-CBC sadly :( $\endgroup$
    – CoffeDev
    Commented Dec 28, 2019 at 11:29
  • 1
    $\begingroup$ CBC mode is just an algorithm (and there is no such thing as AES-Rijndael, it's just AES). Which runtimes are you targetting that are missing update functionality? Because update functionality is an implementation detail. $\endgroup$
    – Maarten Bodewes
    Commented Dec 28, 2019 at 11:35
  • $\begingroup$ @Maarten-reinstateMonica python version(2 and 3) + c + c# + asp.net. I know that it's a very weird combo sadly :( I thought that it would be not worth the effort to try and adapt myself to each language and rely on that their AES library would have the needed functionality, instead i am trying to make an universal solution that works on any plattform and any language as long as we can encrypt a few given bytes with AES in CBC mode in that particular programming language. $\endgroup$
    – CoffeDev
    Commented Dec 28, 2019 at 11:44
  • $\begingroup$ If you have CBC in some form then you can always use the previous ciphertext block as IV and strip off any padded block at the end. So even if you only have single message CBC without update you can implement incremental CBC. Just using CBC should still be the preferred option I guess. $\endgroup$
    – Maarten Bodewes
    Commented Dec 28, 2019 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.