Let's say there are 2 ciphers, $enc$ and $dec$, where:

  1. $enc$ will encrypt the data, and $dec$ will decrypt it.
  2. Both $enc$ and $dec$ must be an AES cipher.
  3. $enc$ and $dec$ will use the same key $k$ and same nonce $n$.
  4. The mode of $enc$ should not be equal to mode of $dec$.
  5. They should not be AEAD ciphers (eg. their modes must not be GCM, CCM etc.)
  6. Both of them should use HMAC (here HMAC-SHA-256).
  7. Encrypt-then-MAC is used by $enc$.

Following the conditions, let's suppose $enc$ is AES-256-CTR-HMAC and $dec$ is AES-256-CFB-HMAC, both initialized with some key $k$ and nonce $n$ where:

$$ k_{len} = 32 $$


$$ n_{len} = 16 $$

Let $data$ represent some data to feed to $enc$, where:

$$ data_{len} = 32 $$

and $ctext$ be the ciphertext, then:

$$ ctext = enc(data) $$

Let $result$ represent the output of $dec$ when $ctext$ is fed to it, ie:

$$ result = dec(ctext) $$

It is known that $data \ne result$, but the hash $h$ of the HMAC for both $enc$ and $dec$ will be the same. Here are my questions.

  1. Is this some design flaw, or is it bound to happen?
  2. If this is not a flaw, then where did my assumption go wrong?
  3. If there was a flaw in my assumption, what is the correct way to use HMAC with AES modes that are not AEAD, eg. CTR, CFB, etc?
  • $\begingroup$ Using different modes for the same cipher can't be more laughable a flaw than practically anything. Please demonstrate what "thinking experiment" value your question has. $\endgroup$ – DannyNiu Dec 31 '20 at 12:12
  • 1
    $\begingroup$ @DannyNiu This simple thought experiment was just a doubt I had while I was studying the algorithms. It is not necessary that there must be some "value" or "reward" associated with it. The answer below helped me clear my doubt about the ciphers in practical spheres, which makes this question have some value (atleast to me). If someone else has the same doubt that I do, they might refer to this. $\endgroup$ – arunanshub Dec 31 '20 at 12:26

In general, we always assume that the mode of operation is known ahead of time. In practice, this should be bound to the key - if the key is only used for a single mode of operation then the above shouldn't happen. However, if the same key is used for multiple modes of operation, then the mode ID should be made part of the ciphertext and therefore included in the MAC. This will prevent the attack you are referring to.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.