If a TLS certificate public-key algorithm is ECC secp384r1 or ECC prime256v1, is it possible to have RSA as a signature algorithm? Or does it use ECDSA for signature? Or can it use any of them (ECDSA or RSA)?

  • $\begingroup$ What Signature are you talking about? Signature by CA or Signature for ephemeral key exchange? (In the first case RFC4492 restricted the CA signature to ECDSA but that was removed in RFC5246. security.stackexchange.com/a/175753 $\endgroup$ – eckes Dec 17 '18 at 2:23

No, RSA cannot be used as a signature algorithm if the certificate has an EC public key in it - you're restricted to ECDSA. You would not be able to verify the signature using the public key after all. The algorithm used must comply with the public key algorithm. However, some keys can be used for different related signature schemes, e.g. RSASSA-PKCS1-v1_5 and RSASSA-PSS (although TLS 1.3 only allows PSS style signatures).

Previously RSA could also be used for session key establishment / authentication using encryption / decryption of the master secret value: the cipher suites starting with RSA_. That use has been deprecated in TLS 1.3, but it is certainly incompatible with EC certificates in versions earlier than TLS 1.2.

  • $\begingroup$ So how do you explain a server's leaf cert. records that contains: ECC prime256v1 for certificate public-key algorithm, and sha256WithRSAEncryption for certificate-signature algorithm, and Digital Signature critical for certificate key uage? The ciphersuite is: TLS_AES_128_GCM_SHA256 and the TLS version is TLSv1.3. One possibility is that the certificate-signature algorithm (certSignatureAlg in OpenSSL terms) means the signature algorithm used by the CA who signed the server's cert. and not the server's cert's signature algorithm. Any idea? $\endgroup$ – user9371654 Dec 16 '18 at 19:12
  • $\begingroup$ See a relevant answer for this post. So, if I want to find server's weak RSA public-keys, I should only care about servers that have RSA as a certificate public-key algorithm (without caring about the certificate signature algorithm as this is the signature made by the CA who signed the server's cert. which can be RSA for a certificate that has ECC public-key). Is this correct? $\endgroup$ – user9371654 Dec 16 '18 at 19:27
  • $\begingroup$ For the certificate question: you could check the structure of the certificate using an online ASN.1 decoder. I commonly use this one. If you get stuck with that you can ask a followup question and include the certificate. There is currently not enough info for me to go on. The last comment seems correct, unless you expect that the CA also has a bad key generation procedure, but I would find it very strange if that was the case. $\endgroup$ – Maarten Bodewes Dec 16 '18 at 21:18
  • $\begingroup$ You can sign a ECC public key with a RSA CA, in that case the signature algorithm in the cert is RSA, however the server cert has still a ECC key which cannot do RSA (and just for completeness, a RSA Server cert can sign a ECDHE key) $\endgroup$ – eckes Dec 17 '18 at 2:12

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.