# Preferred SHA-2 hash algorithm for MGF1 in RSA

Does it make any difference to the security and efficiency if we use SHA-256 or SHA-512 for the Mask Generation Function MGF1 that generates the masking / padding within the OAEP encryption scheme and the PSS signature scheme?

A lot of implementations also seem to sport support for SHA-224 and SHA-384 for MGF1. Does it make any sense to use these truncated hash functions for something that needs to generate a mask of a particular size (close to the size of the key size for RSA)? Or would that simply add unnecessary overhead?

It seems to me that SHA-512 would be more efficient because of the output size alone. However, most implementations seem to default on SHA-1, SHA-256 or the same hash as used to hash the data for PSS.

Since the input sizes are fixed, length-extension attacks are not relevant, so any of the SHA-2 functions reasonably implements the random oracle model assumed by OAEP or PSS via MGF1—even the default of SHA-1 works with MGF1.

Obviously it will cost slightly more to use SHA-224 or SHA-384 than to use SHA-256 or SHA-512 because SHA-224 and SHA-384 are effectively truncations of SHA-256 and SHA-512: to get the same amount of output as (say) twelve SHA-384 invocations each costing a SHA-512 computation, you could pay for a mere nine SHA-512 computations instead. So the truncated options don't provide any benefit. And, of course, SHA-512 than SHA-256 is generally faster on CPUs with 64-bit adders.

All that said, it is hard to imagine that this could substantially affect security or performance since you're about to do a 2048-bit modular exponentiation anyway, which will be the bulk of the cost—but, of course, you're in a better position to do that measurement in your application! (Make sure to choose $$e = 3$$ to minimize time spent in exponentiation.)

• Thanks for your answer. I'm kind of missing the part about output size and SHA-224/386 though. Possibly input block size as well (?). – Maarten - reinstate Monica May 21 at 16:26
• The output size really doesn't matter; you could take a single bit at a time, and repeat it a thousand times, and the security would be unaffected, but obviously it is going to be slightly slower to effectively SHA-512 (say) twelve times via SHA-384 than to compute SHA-512 nine times directly without truncation. The MGF seed—‘input size’—depends on the hash function used to compress the message, not on the MGF hash. – Squeamish Ossifrage May 21 at 16:32
• Let's say that I forget all my knowledge about MGF1, then certainly the input block size makes a difference on how many times the hash function is executed? Of does MGF always perform a single compression of the input? – Maarten - reinstate Monica May 21 at 16:35
• The inputs are pretty much all going to be at most one block long: 256-bit mask seed (unless you're using an obscenely large hash), 32-bit counter for MGF1 blocks, 64-bit input length inside SHA-2 ≤ 512 bits. – Squeamish Ossifrage May 21 at 16:39
• So can I conclude 1. for security it doesn't matter, 2. input block size doesn't matter (much), 3. SHA-224/384 don't make sense, but 4. who cares, modular exponentiation will be much slower than MGF1? I'll quickly do some testing. PS note that e.g. smart cards often have a Montgomery multiplier, but no SHA-2 acceleration. – Maarten - reinstate Monica May 21 at 16:58

A quick comparison in Java of the different algorithms, for 100 000 encryption operations using RSA-4096 in ms (after a startup run to even things out):

RAW    : time taken: 42509 milli-seconds
PKCS#1 : Time taken: 43406 milli-seconds
SHA-224: Time taken: 52437 milli-seconds
SHA-256: Time taken: 44140 milli-seconds
SHA-384: Time taken: 51596 milli-seconds
SHA-512: Time taken: 48293 milli-seconds


So although the differences are not huge, the differences are noticeable. Note that these are the timings for the rather efficient public key encryption, not the private key decryption where the RSA modular exponentiation will take much more of the time compared to the padding.

Of course, above uses OAEP with both the G and H hash algorithm within MGF1 set to the same hash algorithm.

Note that this was done on a dual core i7 in a bit of a make-shift test environment using Java 11. One of the strange things is that the SHA-512 is slower even though it is running on a 64 bit machine.