Using OpenSSL library is there any chance that decrypting AES256-CBC encrypted data with wrong encryption key will succeed without an error?

Asking this because I have read somewhere there is 1/256 chance that decrypting broken data won't produce an error

  • 1
    $\begingroup$ Note that OpenSSL specific questions are often off topic and should be asked on StackOverflow or Superuser. But as this kind of error handling is common to many cryptographic libraries I won't vote to close it. $\endgroup$ – Maarten - reinstate Monica Oct 24 '18 at 16:55

Yes, that's correct, decrypting broken data or data with the wrong key may result in the wrong plaintext without producing an error. Actually, the chance is slightly more than 1/256.

First of all, we need to assume that the data is still a multiple of the block size. If it isn't then it won't decrypt at all.

Furthermore, if the key is correct and the last (and the last few bytes of the first-to-last) cipherblock are correct then the decryption will not fail, even though the data in the first blocks may be incorrect.

If the data is a multiple of the block size then the AES-CBC decryption will never fail. However, OpenSSL uses PKCS#7 padding by default - that is if the high level EVP functions are used instead of the lower level AES functionality. This is true for the command line interface of OpenSSL by the way.

During PKCS#7 unpadding there is a 1/256 chance that the decryption of the last block ends with 01 if the key is incorrect or if the last ciphertext block is incorrect.

However, there is also the chance that the last block ends with 02 02 which is also valid padding. The chance of this happening is however just 1/65536 so it is almost negligible compared to the chance that the last decrypted block ends with 01. This is even more true for 03 03 03, 04 04 04 04 of course. As padding / unpadding must always be performed according to PKCS#7 the chance that a random ciphertext block is accepted is therefore close to 1/256.

An adversary can always change any ciphertext block up to the last as the error propagation properties of CBC are limited to the current block and (part of) the next block. You need to authenticate the ciphertext to avoid this and to avoid padding oracle attacks where those are applicable. Possible options are HMAC over the IV and ciphertext and to switch mode to e.g. GCM.

  • $\begingroup$ Thanks, one more question: I also use AES256-GCM without authentication/AAD. Can it be decrypted successfully with the wrong key? $\endgroup$ – igor.sol Oct 24 '18 at 16:55
  • $\begingroup$ That is: I presume you do call EVP_DecryptFinal and that you do not ignore the return value. From the docs: "When decrypting the return value of EVP_DecryptFinal() or EVP_CipherFinal() indicates if the operation was successful. If it does not indicate success the authentication operation has failed and any output data MUST NOT be used as it is corrupted." $\endgroup$ – Maarten - reinstate Monica Oct 24 '18 at 17:05
  • $\begingroup$ So that should be: No, the AAD data is strickly optional. The verification of the authentication tag will only succeed if the correct key is used. The decryption will succeed but the operation will not report success. The authentication will fail if the key, AAD, IV or ciphertext data is incorrect. Sorry, previous info was not correct... $\endgroup$ – Maarten - reinstate Monica Oct 24 '18 at 17:08
  • $\begingroup$ yes, I have EVP_DecryptFinal_ex call and I don't ignore return value. Thank you for clarification. $\endgroup$ – igor.sol Oct 24 '18 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.