Here is a plain question, all filled with pseudo-real data :)

  1. We have a string: "hello" and two keys for AES: GoodKey and BadKey.
  2. We encrypt our string with GoodKey, and receive something like "lehho".
  3. What will happen if we try to decrypt "lehho" with BadKey? Will decryption algorithm make no sense at some point (like 1/0 makes no sense) OR will it give us some other result, like "goodbye" (which is not original "hello" because of the wrong key)???
  • 2
    $\begingroup$ Note that AES (+ mode of operation) is only able to encrypt bytes, not strings. Similarly, a key must consist of 16, 24 or 32 bytes indistinguishable from random to an attacker. It cannot be "GoodKey" or "BadKey". So please bear this in mind that when thinking about your "pseudo-real" data . $\endgroup$ – Maarten Bodewes Apr 1 '15 at 21:50
  • $\begingroup$ Yes, Maarten, I used strings just for simplicity of example, thought it would be easier to write strings instead of bytes arrays during discussion :) $\endgroup$ – Dima Apr 1 '15 at 22:07

Usually you require a mode of operation for a block cipher such as CBC or CTR to encrypt data. CBC or CTR is then configured with the given block cipher & key. The mode takes an IV or nonce as additional parameter, but I'll leave the IV out of my answer as the IV value is inconsequential if the wrong key is used.

A block cipher such as AES will simply encrypt or decrypt a full block of bytes. For each possible plaintext block there will be precisely one ciphertext block, given a specific key. The result of a bad key will have the same size, but it will be indistinguishable from random otherwise.

A block cipher itself however is not secure and can only encrypt precisely one block - you need the mode of operation too. In that case it depends on the mode. If the mode does not apply padding - e.g. CTR - then the result will be indistinguishable from random, and you won't get an error. If you use CBC (and a padding algorithm instead of ciphertext stealing) then you will probably get a padding error (about 255 in 256 times).

If you always want to receive an error instead of garbled ciphertext or a padding error you need to integrity protect your ciphertext. You can do this by using an authenticated mode of operation such a GCM or by applying a (H)MAC over the ciphertext.

  • $\begingroup$ are you aware of any padding scheme in which a key/cipher mismatch could be detected 100% of the time? $\endgroup$ – 0xbe5077ed Jun 21 at 21:29
  • 1
    $\begingroup$ That would be impossible. You can get close though. You could for instance create a IGE scheme with unbounded padding. In that case any change would be detected at $1 / {2 ^ p}$ where $p$ is the number of padding bits. In general though we don't use padding for this, we use an authenticated cipher or MAC. $\endgroup$ – Maarten Bodewes Jun 23 at 8:38

It depends on the mode of operation. If you use an AEAD mode (which contains a message authentication code, to ensure that the message wasn't tampered with en route to you), then decryption will fail because the MAC is invalid for that message with that key (the whole point of AEAD is that someone without the key can't create a valid ciphertext, so a GoodKey-encrypted ciphertext will be invalid for BadKey). If you aren't using AEAD, you'll just get gibberish -- you can run the decryption algorithm on anything you'd like, it's just a function, but it returns gibberish if you run it on anything that wasn't the result of encrypting with the same key.


An AES decryption with the correct key will return the original message, but an AES decryption with an incorrect key will produce garbage data as an output. The AES cipher itself provides no indication that the key was wrong - there's no point during the decryption at which the algorithm says "hey, wait a minute, this doesn't make sense!" and terminates with an error - it just returns pseudorandom data.

Of course, in practical implementations, we tend to use mechanisms such as padding, which can be used to validate that a decryption was correct. Padding is patterned data which is used to turn arbitrary sized messages into blocks of the correct size for the cipher. Since decryption with a "bad" key results in garbage data, it's highly likely that the padding patterns will be incorrect, which allows you to detect that the decryption operation did not succeed. Of course, this could mean that either the key was wrong, or that the ciphertext (i.e. encrypted) block was somehow damaged.

Normally, in order to provide proper validation of decryption with the correct key, the plaintext message starts with some known value which can be validated. For example, TrueCrypt's volume headers start with the string "TRUE". Additionally, message authenticity and integrity codes (MAC/MAIC) can be applied to to the message to ensure that they are not corrupted or modified.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.