I am currently trying to harden communication over a limited physical communication channel on an embedded system.

The Scenario

I think the system is rather typical for small embedded applications:

  • The embedded systems involved are small microcontrollers without any OS, which can be assumed to be in a secure physical environment. The connection channel is physically accessible to aggressors, they can interfere with messages as desired.

  • Different communication channels are possible: e.g. RS485 or RS232. There is a crude protocol which handles stuff like addressing and crude integrity (e.g. CRC16 for bitflips) on which we can build. Think of it as being a slow TCP.

  • Each communication partner knows a root certificate which can act as a CA and has an own unique certificate(+key) signed by it. Certificates are self signed, elliptic curve prime256v1

  • The mbedTLS library is available for these systems (with DH, AES, ..).

  • Most of the time there is no internet available.

  • The systems can keep time and can create sufficient random numbers

I want to reach privacy, authenticity and integrity.

How it might be solved

A rule in cryptographic protocols is to never implement them on your own. Unfortunately I did not find anything that matches this use case.

I also browsed through existing questions, but nothing provided the full solution i was searching for.

Finally in this answer implementing an own protocol is given as the last option.

This is why I came up with the solution described below - but I am not entirely sure about it. I oriented on this answer on how to authenticate a Diffie-Hellmann Key Exchange, this answer on how to check integrity and this thread giving me some ideas on how to implement such a protocol.

Protocol to secure communication

First thing to do is to randomly generate a symmetric key K. First thing sent is a handshake message. It has the following fields:

Field Description
DH The Diffie-Hellmann key exchange msg gK
Cert The certificate(-chain) of the sending device
MsgIdNonce A random 64-bit value
Signature A signature of the complete message using the private key of Cert

Both devices send (and receive) such a message.

When a message is received the following is done:

  1. Check if cert is signed by the expected CA (has to be in chain)
  2. Check if msg is properly signed with cert and its key

When these checks succeed the sent and received DH gK messages are used to calculate a symmetric key.

From this point on, applications can send messages encrypted with AES-CBC. As messages could be lost relatively often, IV is always in the message.

This is the message format:

Field Description Encrypted
IV Initialy Random IV for AES No
MsgIdNonce Handshake/Last MsgIdNonce + 1 or + 2 Yes
Application Data The data/commands to protect Yes
CMAC truncated to 64 Bit MAC for checking integrity Yes

For a message to be accepted it must

  1. Have the expected MsgIdNonce (1 or 2 above last received one) - preventing replay attacks, allowing to lose 1 message
  2. Have a valid CMAC - preventing attacks on integrity
  3. Be received max. T seconds after the last message - preventing delay attacks

If anything is not as expected, Handshake is sent again, then message is sent again with newly exchanged parameters.


Am I on the right path or should I try something completely different?

Does this provide my security goals or do I miss any obvious vulnerabilities here?

Can this work?

I hope you can help me and other embedded engineers facing similar problems!


1 Answer 1


A rule in cryptographic protocols is to never implement them on your own. Unfortunately I did not find anything that matches this use case.

Have you considered DTLS? That tries to address the same problem (except for time stamps; that could be added by inserting the time stamp in the datagram), and already has things thought through. Another possibility would be IPsec; however DTLS looks closer to addressing the problem you have (unless the data traffic you're protecting is IP traffic).

That said, here are some thoughts on the protocol you outlined:

  • What happens if two sequential messages are dropped (e.g. received incorrectly)? Will the two sides realize what happened, or will they proceed to drop all subsequent messages (because of the 'nonce has to increment no more than 2' rule)?

  • Presuming they attempt a rekey, how is this coordinated? If one side thinks they haven't rekeyed yet, and the other side thinks they are, what will happen to messages encrypted with the old key? We find it generally useful to include a 'which key this is encrypted under' tag with the ciphertext (in DTLS, this is the 'epoch').

  • Can your communication channel reorder messages? I suspect it can't in your case, but if it can, you should think about how that should be handled.

  • What happens if someone injects a previous (valid) handshake message? How will that confuse things?

  • In the data encryption path, you use CBC encryption and CMAC integrity. While this is a workable solution, this is prone to various padding attacks if they aren't integrated correctly. Current fashion is to use an AEAD mode (such as GCM or Chacha20/Poly1305) which does both.

  • Is the encrypted traffic unidirectional or bidirectional? If traffic goes both ways, do you use the same set of keys to protect both sets of traffic?

  • "If anything is not as expected, Handshake is sent again, then message is sent again with newly exchanged parameters." - how does that work? If the receiver decided it didn't like the message, how does the sender know to resend it?

  • $\begingroup$ I will take a look at DTLS. Up to now I only thought of TLS, but that did not seem to work with my transport. Mbedtls makes it easy to change the transport to UART/RS485, I will have to evaluate how this works out in practice. $\endgroup$
    – mile4712
    Commented Jan 5, 2021 at 8:01
  • $\begingroup$ If I can't use it for some reason I will work your questions in and leave it here. Here are my first thoughts: Seems like i was trying to build a AEAD mode myself! - I would change that to GCM. The channel is bidirectional, and I would use the keys for both directions. MsgIdNonce is valid only for 1 Direction. $\endgroup$
    – mile4712
    Commented Jan 5, 2021 at 8:01
  • $\begingroup$ Applications have to handle losing messages every once in a while - so two sequential messages dropped should resend a handshake and new messages with old key would be lost. As you noted, receivers can't retrigger a resend - so that message would be lost as well. For rekeying I would send a Handshake whenever I received one - and lose messages with the old key. Injecting an old handshake message would result in broken communication (MsgIdNonce would not fit for proper messages), which would result in a new Handshake. You are right, this channel cannot reorder messages. $\endgroup$
    – mile4712
    Commented Jan 5, 2021 at 8:02
  • $\begingroup$ @mile4712: if you do use GCM with the same key in both directions, you should take some thought as to how to make sure that both sides don't send a message with the same nonce; it can be easy (e.g. one side uses even nonces, the other side uses odd ones), however it should be done. On the other hand, GCM doesn't need randomness for its nonces; you can use a simple counter to generate them $\endgroup$
    – poncho
    Commented Jan 5, 2021 at 17:51
  • 1
    $\begingroup$ @mile4712: the other thing to consider (which I forgot to mention) is "what happens if both sides send out initial handshake messages at about the same time? Is there a potential race condition there that can confuse things?" Depending on how you structure things, it might be benign; however you should think this through $\endgroup$
    – poncho
    Commented Jan 5, 2021 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.