4
$\begingroup$

I'm writing a client-server encryption scheme for homework, and I've stumbled upon what might be a fatal blow to my current implementation.

When using AES-256-CBC on the two sides, it's important that the client's encrypt context and the server's decrypt context remain in sync. However, my protocol allows for rejection of messages if they fail an HMAC test.

If a message were to fail the HMAC test, then the two contexts fall out of sync because the state of the context was polluted by the garbage ciphertext. All subsequent decrypts will fail. Is there any way to rollback a decryption, i.e. bring it back to the state in which it was before the decrypt happened? Alternatively, is there any way to copy an decrypt context?

The same goes for the client when it receives word of the failure. It's going to want to roll back its encryption.

Should I use something other than CBC mode?

Improve this question
$\endgroup$
1
  • $\begingroup$ You could use EVP_CIPHER_CTX_copy to create a copy of the previous context in the respective applications and maybe use that to do the subsequent crypto procedures, when a MAC fails. I haven't tested it, but please do :) The documentation is here: openssl.org/docs/manmaster/man3/EVP_CIPHER_CTX_copy.html $\endgroup$
    – Aravind A
    Commented Oct 18, 2023 at 5:43

1 Answer 1

Reset to default
2
$\begingroup$

You should packetize, or recordize, the messages into discrete units of a size reasonable for your protocol. Each record should be independently encrypted with its own IV. As an example, look at TLS 1.2 record or IPsec ESP packets. If a record is corrupted it can be discarded without impacting other records.

EDIT:

As pointed out by @poncho, TLS incorporates an implicit sequence number in the MAC calculation. Thus this may not be a good example. He also points that DTLS includes an explicit sequence number, and thus may be a better example.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So essentially switch to counter mode and use the sequence number in the HMAC? $\endgroup$
    – alexgolec
    Commented Feb 12, 2012 at 16:27
  • $\begingroup$ @Alex: No, simply send the initialization vector for each message with the message itself. Of course, this only works if the attacker inserts only whole packets, i.e. for a packet-based protocol. For stream based protocols it is usual to break the connection when a packet comes with wrong MAC. $\endgroup$
    – Paŭlo Ebermann
    Commented Feb 12, 2012 at 17:30
  • 1
    $\begingroup$ @GregS: While IPsec ESP is a good example, TLS is not. A corrupted TLS record shuts down the connection; even in TLS 1.1 (where the IV is included in the record), there is an implicit reliance on a sequence number, which must also remain in sync between the sender and the reciever. Perhaps you meant the rather more obscure DTLS (which has the sequence number in the record as well)? $\endgroup$
    – poncho
    Commented Feb 12, 2012 at 22:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.