Do I need to use AE/AEAD in my particular scenario? Or is it redundant?

Question

This is my scenario:

1. A small structured text message (1-2kB) is to be sent over an insecure channel.
2. Everything except for the header portion (which contains a title, timestamp, category and tags) is confidential.
3. The message as a whole has integrity and non-repudiation as additional requirements.

This is how I intend to go about it after having done some crypto-related research:

1. Encrypt the message with AES-CBC/OFB/CTR (key is pre-shared). The IV is added to the front of the ciphertext.
2. Add the non-confidential portions to the output from (1).
3. Make a digest-hash(SHA512 or RIPEMD160) of the output from (2) and encrypt it with the message author's private key (public key is pre-shared). Add that to the output from (2).

4. Send it over the insecure channel.

   <---IV---><---AES Ciphertext---><---Header---><---Signature---> 
   

Question:

I know that MAC algorithms and digest algorithms were designed for different purposes, but in this case, my digest is not merely slapped onto the payload, it is asymmetrically encrypted.

So, would the choice of AES-GCM/CCM/EAX have added any additional security? Does my scheme have any obvious vulnerability that I overlooked?

Any help is greatly appreciated.

EDITS:

1. Fixed a typo. OBC $\rightarrow$ OFB.
2. I did not realize that the hashing algorithms are already built into all the major signing algorithms; I thought you had to do it yourself. Thanks for clearing that up.

NOTE: I would have chosen Ricky D.'s advice, but comments cannot be chosen as answers.

Answer 1

In general signature creation contains the hashing part within the algorithm. A signature algorithm may also contain a padding mechanism such as PKCS#1 v1.5 or PSS for RSA. Finally it contains a one-way trap door function (modular exponentiation within RSA). Encryption has other requirements, and uses a different padding mechanism.

Basically you are encrypting a plaintext, and signing all the data including IV within the message. I don't see the need for either a MAC or an authenticated cipher; the signature is perfectly fine as authentication tag. That does not mean that your current scheme does not have weaknesses:

• it does not provide forward secrecy (i.e. if your secret key is compromised, all the messages can be decrypted)
• you have to keep both a private key and share a secret key
• asymmetric operations are expensive (with regards to CPU) and you need a private and public key operation per message
• no signer anonymity (eavesdroppers can prove that messages were sent by a specific sender)
• as others have pointed out, encrypt-then-sign may have vulnerabilities you should be aware of in your protocol

Those drawbacks may be resolved by a careful choice of a TLS cipher suite (if you just require transport security).

Finally you may want to think about how you are going to setup key management.

Answer 2

A symmetrically encrypted hash is not a secure MAC. You should use either an authenticated encryption scheme or a secure MAC in encrypt-then-MAC.

With asymmetric encryption, it may be secure – "encrypting" with the author's private key means you are actually signing the message which is fine. However, you need to use the actual asymmetric primitive, not some high level hybrid encryption, because then the hash would actually be encrypted symmetrically. That means security may also depend on your getting padding right etc. so you should use a high level signing interface, if the libraries you use have one.

Had you meant OCB mode, that would be one option. It should also allow additional data that is only authenticated. However, there are some patent concerns and possible attacks, see the Wikipedia page for some references. Since you meant OFB, it is not a very good mode to use – it's basically worse than CTR in many ways, but with no advantages like CBC or CFB have.

Another option is to use HMAC together with AES in CTR mode (CBC would also work, but is easier to get wrong). That lets you use any of the hash functions you have available. My choice would be SHA-256 if you have that. This is probably the easiest way available to you, considering you already have symmetric keys shared.

Answer 3

encrypt it with the message author's private key

This statement makes me uncomfortable. Normally, in asymmetric cryptography, one encrypts with the public key and signs with the private key. Did you mean “sign it with the message author's private key”? Otherwise, I would not accept your protocol without a clear, detailed explanation of what encryption mode you're going to use, and why you are encrypting rather than signing.

If you used a signature, what you would be doing would be encrypt-then-sign. You should read Should we sign-then-encrypt, or encrypt-then-sign?. D.W.'s answer mentions a potential flaw of encrypt-then-sign; whether it's an issue depends on how you verify the message. Anyone can take the ciphertext from stage 1 and tack on their own cleartext header and their own signature. On the recipient side, it is vital that you reject messages where the signature is not made by the party holding the private key: otherwise the recipient may end up decrypting a message on behalf of an adversary.

Given that the ownership of the private key has to be tied with ownership of the symmetric key, I can't see any benefit of your scheme over AEAD. Using AEAD has the advantage that professional cryptographers have pored over the subtle implications of the protocol. A secondary benefit is that symmetric encryption is faster.

One good reason not to use AEAD is if you don't want the recipient to be able to forge messages. In that case, with the caveat above about verifying that signatures match, I think that encrypt-then-sign is ok.