I am working on an experiment to compare the efficiency of brute-force attacking block ciphers like AES and DES with reduced key sizes for different programming languages and library implementations. However, I am restricting it to using a ciphertext only attack, so I will generate 1 block of random plaintext, encrypt it, and then the program will attempt to decrypt the ciphertext by brute-forcing the key.

However, I am getting stuck when thinking about how to verify the attack has succeeded. If the plaintext I was using was a long enough article of standard English, I could do frequency analysis, and use this as a measure of confidence that the attack succeeded. However, with a purely random plaintext, I am not sure how to distinguish that any P' produced as the result of an attempt of the brute-force attack is the correct plaintext or not. Essentially, without knowing the plaintext, how do I know my attack has succeeded?

This led me to the topic of Authenticated Encryption, which I am very new to, and I'm not sure that I fully understand it. From my understanding, I could use Encrypt-then-MAC or MAC-then-Encrypt. If I were to do MAC-then-Encrypt, my MAC would be included in the decrypted plaintext, so I could verify that my attack had succeeded by checking that the MAC and plaintext correspond for any P'. I've seen mention that Encrypt-then-MAC is more secure, but I am not sure entirely sure why, and I am not sure if Encrypt-then-MAC would allow me to verify whether my attack has succeeded.

Any guidance in either AE or simply how to verify whether my attack has succeeded is greatly appreciated, thanks.

  • $\begingroup$ What is the mode of operation? If CBC you can use the padding as done for DES challenges. We have a canonical question Should we MAC-then-encrypt or encrypt-then-MAC?. If encrypted then MAC'ed then you cannot use the MAC on the possible plaintext since there is no MAC over it. $\endgroup$
    – kelalaka
    Nov 29, 2023 at 19:20
  • $\begingroup$ We almost never assume random plaintext, why would you? Ciphers need to be secure regardless of the message both in theory and in practice. Why wouldn't you able to verify the correctness of the key using encrypt-then-MAC? In both cases you'd only have the MAC (assuming that other information such as padding is not available). $\endgroup$
    – Maarten Bodewes
    Nov 29, 2023 at 20:44
  • $\begingroup$ It does seem that you are unnecessarily thing one hand behind your back by assuming random plaintext. It is your choice of course - but a more usual approach would be to calculate the entropy of the decrypted block. If it is noticeably less than 8 bits per byte, the decryption was successful. You don’t need to assume English and you don’t need large amounts of text. $\endgroup$ Nov 30, 2023 at 8:00

2 Answers 2


The question linked from @kelalaka is to answer the merit of Encrypt-then-MAC.

and I am not sure if Encrypt-then-MAC would allow me to verify whether my attack has succeeded.

That's easy. The verification of MAC happens before decryption in these ciphers, since it's first "encrypt" then "MAC".

As to do it practically, you can take any authenticated encryption implementation for any mode (or special construction such as ChaCha20-Poly1305), and remove the unneeded part (e.g. removing decryption part if it's after MAC verification).


Use implementations as is, and examine the return status on whether the decryption was successful - most implementations would have to return such status, so that they (the implementations) can be used in higher-layer protocols.


I confirm the intuition in the question: with a uniformly random plaintext and block ciphers (like AES or DES) in any of their common modes of operation for pure encryption (CTR, CBC, OFB, CFB, ECB) and no padding, there is no way to make a ciphertext-only attack, regardless of attack method, size and number of ciphertexts, computational effort, and block cipher. A mathematical proof follows from the fact that there are exactly as many ciphertexts as there are plaintext of a given size, and encryption followed by decryption succeeds for any plaintext.

If we add padding, that might or not make it possible to narrow down the possible keys, because it effectively makes the end of the plaintext non-random. That's possible e.g. for PKCS#7 padding.

When doing MAC-then-encrypt or encrypt-then-MAC with block ciphers for the MAC part, it is important to use independent keys for the MAC and the encryption part (if not, it's sometime possible to breach the integrity that the MAC aims to provide, including in ciphertext-only active attack). Therefore independent keys for encryption and integrity is the best practice, irrespective of how the MAC is done, with the exception of modes that integrate encryption and integrity, e.g. GCM.

With independent keys for encryption and integrity, or when these keys are derived from a master key using a good key derivation function, or when the encryption and MAC are not based on the same block cipher (e.g. the MAC is HMAC):

  • in encrypt-then-MAC, there is no way to break the encryption, even if we manage to find the MAC key.
  • in MAC-then-encrypt, if we know the MAC key, we can rule out some encryption keys for a given MAC-then-encrypted message. With enough such messages, we can narrow down on a single key. That's also possible with artificially small unknown keys for MAC and encryption, and enough MAC-then-encrypted messages. We need roughly as many messages as there are possible combinations of MAC and encryption keys.

In most AEAD modes, there is a single key, the authentication does allow to rule out most keys in a brute force attack, and that's the only way for random plaintext.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.