RSA hybrid decryption via PKCS#11 - unwrap or decrypt?

Question

I have an application that does RSA hybrid encryption/decryption – i.e., messages are encrypted with a fresh AES key, which is then itself encrypted with RSA-OAEP and sent with the message. Decryption then does the reverse.

I want to now provide support for hardware security modules (HSMs) via PKCS#11. (The application is written in Java, so this will actually be via some wrapper such as the default SunPKCS11 library). I can see two options for implementing decryption:

1. Use the HSM to decrypt the temporary AES key and return the raw bytes of that key to my application, which then does the AES decryption in-memory.
2. Use the HSM to “unwrap” the AES key as a (sensitive) session object and then use the HSM to perform the AES decryption.

My instinct is that option 1 will generally be faster than option 2 on a server with AES-NI instructions. As the AES keys are unique to each message, it also doesn’t seem less secure - the AES key is therefore no more sensitive than the message itself.

Are there any reasons to prefer option 2? My only thought is that if a customer has spent a lot of money on a FIPS-compliant HSM then they may want all crypto to be performed on the device for compliance reasons. Is that a likely scenario?

Answer 1

The AES is just used to protect the data. And although keeping the AES key in the HSM is more secure, you need to remember that there is little reason to protect it any better than the data itself.

However, if you don't need the speed, why not go for the HSM option? At least you'd be reasonably sure that the key is secure, that there are no side channel attacks and that the algorithm will operate as designed (e.g. with regards to random generation of the IV, self tests, destruction of the key etc.).

So use AES in software if you require the speed, HSM resources or low latency. Otherwise use the AES key in the HSM if you don't require it.