How to enforce data privacy, integrity and authenticity of distributed firmware images for an embedded device?

Question

I'm developing a bootloader for an embedded device, with which it should be possible for customers (i.e. device owners) to update the device's firmware. It is required that:

1. the firmware is distributed encrypted
2. the firmware's integrity is protected
3. the firmware is authentic, meaning that it was released by us (the manufacturer).

Based on my limited knowledge about cryptographic schemes, I came up with the following potential solution (see below for explanation):

• For enryption and data integrity the firmware would be encrypted using AES256 in EAX mode using a key, which is shared between the manufacturer and the bootloader on the device, in addition to a nonce (i.e. initialization vector), which is encrypted (signed) using a private RSA key
• The AES encrypted firmware and the RSA encrypted nonce are then distributed to the customer
• The bootloader on the customer's device would then decrypt the nonce with the "public" RSA key (to ensure authenticity), and decrypt the firmware using the AES key and the nonce

Does this constitute a valid scheme for a secure bootloader (in terms of data privacy, integrity and authenticity)?

I am aware that using AES in EAX mode already provides data authenticity, however I would like to make sure that authenticity is still ensured in case the AES key is compromised.

Answer 1

First, if you mean to sign something, you should use a signature algorithm, rather than "RSA encryption" with private key, which on its own may be very weak.

Now the main issue here is that the RSA part is only authenticating the nonce. Anyone who knows the AES key can just encrypt an arbitrary firmware image with a nonce they've already seen. This allows them to reuse the earlier signature. So there's no guarantee of authenticity if the AES key is compromised.

To fix this you could sign the whole encrypted image – nonce, authentication tag and all. With a strong signature algorithm (e.g. RSASSA-PSS) using a strong hash function, there is no way even knowing the AES key to encrypt another image with the same signature.