1
$\begingroup$

I am using CAPI Engine in OpenSSL and I did some test. When I use TLS 1.0 or 1.1, during handshake and RSA signing, PKCS padding is chosen. When I use TLS 1.2, RSA signing uses PSS padding.

  • What affects the choice among paddings?
  • How client knows what padding was used?
  • When is padding assigned to RSA structure?
$\endgroup$
1
$\begingroup$

Q: I am using CAPI Engine in OpenSSL and I did some test. When I use TLS 1.0 or 1.1, during handshake and RSA signing, PKCS padding is chosen. When I use TLS 1.2, RSA signing uses PSS padding.

Funny, a quick lookup of TLS 1.2 contains the following:

Note that there are certificates that use algorithms and/or algorithm combinations that cannot be currently used with TLS. For example, a certificate with RSASSA-PSS signature key (id-RSASSA-PSS OID in SubjectPublicKeyInfo) cannot be used because TLS defines no corresponding signature algorithm.

As I didn't find any RFC's that counteract this, it seems impossible that TLS 1.2 uses PSS for authentication at all.

Now, on the other hand TLS 1.3 specifies that you should use PSS, which is not that surprising as TLS 1.3 very deliberately uses up to date cryptography at the cost of backwards compatibility.

For TLS 1.0 a very specific structure with both SHA-1 and MD5 is used. This standard predates SHA-2 and therefore it uses a more secure construct using two hashes that are separately pretty insecure. It also may use RSA encryption using PKCS#1 v1.5 padding as PKCS#1 v2.0 hadn't been introduced either. I don't see any changes for TLS 1.1 for RSA signing / encryption. Note that TLS 1.1 and 1.2 were mainly incremental updates of TLS 1.0 rather than the significant rewrite that TLS 1.3 is.

So basically, the use of RSA padding in TLS is directly tied to the standard itself. OpenSSL will try and follow that standard.


It might be that if you have a certificate that identifies the public key algorithm as PSS, see here. In that case an implementation may decide to go with that single choice, even though it is prohibited by the TLS 1.2 standard - I guess the correct response would be to throw an error, but may be a warning is generated instead.

| improve this answer | |
$\endgroup$
  • $\begingroup$ 8446 4.2.3 at the end (on p22) says a 1.3-capable implementation negotiating 1.2 must support PSS signatures. Of course if the peer supports 1.3, there should be no good reason not to negotiate 1.3, and if the peer only supports 1.2 it probably won't use PSS. $\endgroup$ – dave_thompson_085 Apr 5 at 6:27
  • $\begingroup$ And for completeness, in 1.2 client knows what signature padding is used from the ''algorithm' field in 'digitally-signed' in ServerKeyExchange (and server knows client from CertVerify if used); in 1.3 this field occurs directly in CertVerify for both directions. $\endgroup$ – dave_thompson_085 Apr 5 at 6:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.