How to use separate keys for signing and encryption

Question

I read that the same key should not be used for both certificate signing and encryption.

I wonder if this is not a problem with PKI/TLS/SSL, the reason being asymmetric encryption is used to transmit the public key (via the certificate). The public key is used to encrypt a browser generated session key. The session key is used for symmetric encryption of the actual traffic.

Is that how it works and does that mean it is okay to sign both with the same key?

How would I go about using separate keys?

Answer 1

In TLS 1.2 the Client proposes a list of suites of primitives they are willing to use for TLS. This takes place in clear in the ClientHello message.

You could examine that list and decide one of three things:

1. The list has a nice modern Forward Secret Diffie-Hellman-style key agreement method you're comfortable using with RSA signatures. You present Certificate #1 for RSA Key #1 and you use the corresponding Private Key to sign stuff in TLS. Certificate #1 says this RSA key is suitable for this purpose (specifically it should set the Key Usage bit 0 digitalSignature).

2. The list says the client is willing to do RSA key agreement. You present Certificate #2 for RSA key #2 and you use the corresponding Private Key to decrypt the key agreement message from the client. Certificate #2 says this RSA key is suitable for this purpose but not for signing things (probably setting bit 2 of Key Usage aka keyEncipherment -- it should definitely not set bit 0 digitalSignature).

OR 3. You can't accept any of these methods and give up.

Having chosen to behave this way you can now also do TLS 1.3, you should always use Certificate #1 (and the corresponding key) because in TLS 1.3 there is no RSA key agreement mechanism since it is considered dangerous. Because you are using separate keys, a client which is careful about validating Key Usage information can discern that it should not trust Certificate #2 in TLS 1.3 (or doing DH key agreement in TLS 1.2) and so if an attack on your decryption in scenario #2 above is successful those clients are protected and only clients willing to undergo scenario #2 themselves are affected.

Answer 2

First of all, you cannot use a private key for signing and encryption, you'd use it for signing and decryption.

In PKI/TLS/SSL asymmetric encryption is not used to transmit the public key when RSA session key agreement is used. The public key is within a (signed) certificate, the private key stays at the server.

Once trusted, the public key can be used to encrypt the master secret, which is used to derive the session keys.

---

Is that how it works and does that mean it is okay to sign both with the same key?

So no, that's not how it is used. That said, sometimes the same private key is used for signing or decryption within TLS. This happens when both asymmetric (DH) key agreement and RSA encryption are used within the enabled ciphersuites. Within DH key agreement the private key authenticates the parameters rather than decrypting the master secret directly.

Although separating different uses of keys is good practice with regards to key management. That doesn't mean that using the keys for authentication and decryption will suddenly destroy all security however. Using different keys for e.g. long term signatures (e.g. document signing) and real-time authentication makes a lot of sense.

To use different key pairs it is normally sufficient to use different certificates / certificate chains. The (TLS) software should pick up on the correct key usage and should therefore not use a certificate / private key pair for the wrong purpose - but that is of course implementation specific. In other words, don't assume but test you specific TLS configuration.