1
$\begingroup$

I am wrapping the RSACng class from the .Net API. Whilst unit testing, I run checks against an array of possibilities:

  • Key Size = 1024, 2048, or 3072 bits
  • Padding Mode = PKCS v1, or PSS (=PKCS v2?)
  • Hash Algorithm = SHA-1, SHA256, SHA512

I test for all 3*2*3=18 possible configurations. All tests pass except one combination:

  • Key size = 1024 bits
  • Padding Mode = PSS
  • Hash Algorithm = SHA512

Should this be expected, and how could have I anticipated it? So far, Google has not been my friend.

I also ran a similar batch of unit tests for RSA encryption, and noted a 1024 bits key is incompatible with OAEP-SHA1 and OAEP-SHA256 padding modes. I suspect the key is not large enought, but I was again unable to read some info as to why this was happening.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Having about 80 bits of security, using 1024 bit RSA is also incompatible with most key size requirements, and mixing it with a hash function that supplies close to 512 bits of security in this setting makes little to no sense. $\endgroup$
    – Maarten Bodewes ♦
    Commented Feb 7, 2020 at 21:52

1 Answer 1

Reset to default
4
$\begingroup$

Yes, this is expected, if the RSASSA-PSS signing code/test uses salt the width of the hash, which is customary. In that case, a $h$-bit hash (with $h$ multiple of 8) requires an RSA public modulus at least $2h+10$ bits (see justification in comment). That's semi-clearly stated in PKCS#1 v2.2, section 9.1.1, condition on enBits, with actual test in step 3.

That's because the message representative needs to convey $h$ bits for the message hash, $h$ bits for the salt, $8$ bits for a one-byte identifier¹, so we are at $2h+8$. Then there is a 01h byte stuffed on the left, which requires one additional bit. And the public modulus needs to be at least one bit larger so that the value obtained is always less than the public modulus.

In PSS, the salt length can be made a parameter, rather than fixed to the hash width as usual. Also, it can optionally be deterministic or fixed, rather than random as usual. But far from all APIs allow that.

There are similar limitations for RSAES-OAEP (without flexibility: the tag's hash always is as wide as the hash, even when the tag is empty/absent, as it often is).

¹ That identifier BCh plays a role when using PSS padding for Rabin signatures or when it is used $r\to\min(r^d\bmod N,N-(r^d\bmod N))$ instead of the textbook RSA private key operation, in order to allow immediate textbook-RSA encryption of the signature with any public key with public modulus of the same size as the one used for signature.

Improve this answer
$\endgroup$
9
  • $\begingroup$ I've got the OAEP calculations for size in handy bytes over here. Note the N/A for RSA-1024 with OAEP using SHA-512. $\endgroup$
    – Maarten Bodewes ♦
    Commented Feb 8, 2020 at 4:32
  • $\begingroup$ @MaartenBodewes that's interesting because .Net allows me to combine RSA1024 & OAEP-SHA512, but not to combine RSA1024 & OAEP-SHA1 or SHA256 (allow = does not throw an error). I understand some combinations might make little sense, which is partly why I am testing all possible scenarios: I wanted to know if my wrapper requires to "validate" the configuration (key length & hash length) another programmer would set up whilst using the wrapper. (sort of a last resort layer of security) $\endgroup$
    – Ama
    Commented Feb 10, 2020 at 16:54
  • $\begingroup$ I'll be short: If you want to make sense of anything, the .NET crypto API is not a good starting point. It does some things correctly, many other things are unspecified or just badly designed. $\endgroup$
    – Maarten Bodewes ♦
    Commented Feb 10, 2020 at 16:56
  • $\begingroup$ About the salt length: "rather than fixed to the hash width as usual". Is the hash width recommended or optimal in some way? If not, why has it become the "usual" choice? OpenSSL by default seems to choose the maximum salt length when signing. $\endgroup$
    – Reinier Torenbeek
    Commented Jul 27, 2021 at 17:30
  • 1
    $\begingroup$ @Richard Kettlewell: ah, I got it: there's this 0x01 byte that I did not account for, which requires an extra bit. $\endgroup$
    – fgrieu ♦
    Commented Apr 6, 2023 at 10:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.