# RSA PKCS#1 OAEP Padding Schema - padding 0 for first octet?

In this document, it shows the EM will have one zero octet padding prefixed.

                     +----------+---------+-------+
DB = |  lHash   |    PS   |   M   |
+----------+---------+-------+
|
+----------+              V
|   seed   |--> MGF ---> xor
+----------+              |
|                   |
+--+     V                   |
|00|    xor <----- MGF <-----|
+--+     |                   |
|      |                   |
V      V                   V
+--+----------+----------------------------+
+--+----------+----------------------------+


However, in this document, page 37, it does not have this 00 octet.

For the standard (or the implementation of openssl) RSA PKCS#1 OAEP padding, which one (with 00 octet or without octet) is the standard/correct?

In fact, I've done the following tests:

1. I use RSA_public_encrypt(RSA_PKCS1_OAEP_PADDING) and find out the max message length is 214, which should be yielded by 256-20*2-2(k - 2hLen - 2).However, per https://www.openssl.org/docs/man1.0.2/man3/RSA_private_decrypt.html, it mentions "EME-OAEP as defined in PKCS #1 v2.0" so I think it should be based on rfc2437. But the max length for rfc2437 is "emLen-1-2hLen", which is a conflict?

2. I use RSA_public_encrypt(RSA_PKCS1_OAEP_PADDING) to encrypt a message(8 bytes only) and then use RSA_public_decrypt(RSA_NO_PADDING) to decrypt it to see the output. I notice there always be a leading 00 octet padded. This suggests it is based on rfc3447(v2.1). I also have no idea about this.

• It matters what version of PKCS1 you use because 2.0 used one notation and 2.1 and 2.2 used a different but functionally equivalent notation. In 2.0 (rfc2437) EME-OAEP-ENCODE is invoked with emLen = k-1 where k is the size of the modulus n=pq, and the 'message' is limited to emLen-2hLen-1 = k-2hLen-2 . EM is then k-1 octets, but when converted by OS2IP in step 2 of 7.1.1 it results in the 'k-size' integer equal to 00 || EM, and on decryption step 4 of 7.1.2 enforces this. Compare crypto.stackexchange.com/questions/40032/… . Dec 31 '20 at 23:24

According to RFC 2437...

• RSAES-OAEP can operate on messages of length up to $$k-2-2\cdot hLen$$ octets, where $$hLen$$ is the length of the hash function output for EME-OAEP and $$k$$ is the length in octets of the recipient's RSA modulus.
• The padding string $$\text{PS}$$ in EME-PKCS1-v1_5 is at least eight octets long, which is a security condition for public-key operations that prevents an attacker from recovering data by trying all possible encryption blocks.
1. Generate an octet string PS consisting of $$emLen-||M||-2\cdot hLen-1$$ zero octets. The length of $$\text{PS}$$ may be $$0$$.
• concisely, the length of zero padding will be $${emLen-||M||-2\cdot hLen-1}$$

Running with OpenSSL;

openssl rsautl -decrypt -inkey priv2.txt -in cipher.txt -raw -hexdump

and the first line of the output is;

0000 - 00 25 2b 2c e4 b5 a8 6c-a1 d1 cc bd 0b 26 d2 9c .%+,...l.....&..

You can see that the first byte above is 0x00

• Quick question, why not reference the newer RFC 8017? Dec 31 '20 at 6:18
• Thanks SSA and DannyNiu. My confusion comes from the following: Per openssl.org/docs/man1.0.2/man3/RSA_private_decrypt.html, for the OAEP, it states it is "EME-OAEP as defined in PKCS #1 v2.0 with SHA-1,...". Therefore, it should be based on RFC 2437. However, when I RSA_private_encrypt(RSA_PKCS1_OAEP_PADDING) and then RSA_private_decrypt(RSA_NO_PADDING), I always see a leading 00 octet is padded. Any idea?
– CHL
Dec 31 '20 at 7:33
• @CHL, Are you getting 00 with any arbitrary plaintext? Could you provide us your sample plaintext and openssl command you use for encryption and decryption?
– SSA
Dec 31 '20 at 10:13
• you will get 00 always padded at the beginning of padded string. Is that you are asking?
– SSA
Dec 31 '20 at 10:47
• Yes. I always get 00 when I use RSA_private_decrypt(RSA_NO_PADDING).
– CHL
Dec 31 '20 at 11:50