TLS 1.2 Server Key Exchange problem

Question

Below the Wireshark snapshot of the handshake compliant to the TLS 1.2.

The position 1 displays the enum ServerECDHParams where I can use any curve having the own ID RFC8422. I'm free for example to use Edwards, Brainpool or NIST curves having an ID assigned by IANA.

Please correct me when I'm wrong but at the position 2 I have to provide the value from the enum SignatureScheme which defines not only the signature algorithm (ECDSA in this case) but also the hash and the curve. What value should I then use if the ECDSA certificate contains a signed public key based on for example brainpoolP256?

The migration to TLS1.3 is not an option because the task was to extend the proprietary legacy server side TLS1.2 stack implementation to use the ECC.

Not using of brainpool is also not an option as this is a customer requirement.

Full specification of the domain parameters in the Server Key Exchage is hardly supported by any embedded TLS 1.2 client.

Thank you in advance

Answer 1

You're wrong. This is a difference between TLS 1.2 and 1.3. In 1.2 SignatureAndHashAlgorithm identifies only the algorithm (not curve) and hash. In 1.3 SignatureScheme does identify the curve for ECDSA, and the certificate OID for RSA-PSS. See the next to last para on page 44:

[1.3] ECDSA signature schemes align with TLS 1.2's ECDSA hash/signature pairs. However, the old semantics did not constrain the signing curve. If TLS 1.2 is negotiated, implementations MUST be prepared to accept a signature that uses any curve that they advertised in the "supported_groups" extension.

(although in 1.2 and lower before RFC7919, that extension was elliptic_curves not supported_groups!)

Wireshark is showing both decodes: the single, combined SignatureScheme name for 1.3, and the separate hash+signature names for 1.2, and only the latter apply in your case. This is arguably a bug: some other places this field occurs it could be ambiguous whether to use the 1.2 or 1.3 interpretation, but ServerKX can (occur and) contain this field only in 1.2 so it should have the 1.2 interpretation.

And the brainpool curves are designed so that standard (X9.62) ECDSA works with them. Edwards curves as such cannot, but (IINM) they can be converted to Weierstrass curves which can.