Given this description from RFC 4880 sec 5.1:
The value "m" in the above formulas is derived from the session key as follows. First, the session key is prefixed with a one-octet algorithm identifier that specifies the symmetric encryption algorithm used to encrypt the following Symmetrically Encrypted Data Packet. Then a two-octet checksum is appended, which is equal to the sum of the preceding session key octets, not including the algorithm identifier, modulo 65536. This value is then encoded as described in PKCS#1 block encoding EME-PKCS1-v1_5 in Section 7.2.1 of [RFC3447] to form the "m" value used in the formulas above. See Section 13.1 of this document for notes on OpenPGP’s use of PKCS#1.
and this guess at what the pronoun this in the passage means, what is the proper way to format m? The description seems somewhat vague at where everything goes.
13.1.1. EME-PKCS1-v1_5-ENCODE
Input:
k = the length in octets of the key modulus
M = message to be encoded, an octet string of length mLen, where
mLen <= k - 11
Output:
EM = encoded message, an octet string of length k
Error: "message too long"
1. Length checking: If mLen > k - 11, output "message too long" and
stop.
2. Generate an octet string PS of length k - mLen - 3 consisting of
pseudo-randomly generated nonzero octets. The length of PS will
be at least eight octets.
3. Concatenate PS, the message M, and other padding to form an
encoded message EM of length k octets as
EM = 0x00 || 0x02 || PS || 0x00 || M.
4. Output EM.
Given this data:
ElGamal modulus p has 512 bits -> 64 octets
Symmetric key algorithm = AES128 = 0x07
secret key = 0x0123456789abcdef0123456789abcdef
checksum = 2 * (0x01 + 0x23 + 0x45 + 0x67 + 0x89 + 0xab + 0xcd + 0xef) = 0x780
At first, i thought it was: "0002" + random_non_zero_data + "00070123456789abcdef0123456789abcdef0780", but when i try to decrypt the data using a real PGP program, it says the key checksum failed.
Then, I get contradictory data from PGPdump: m = sym alg(1 byte) + checksum(2 bytes) + PKCS-1 block type 02, so i tried "070780" + "0002" + random_non_zero_data + "0123456789abcdef0123456789abcdef", but that didnt work either ("exception decrypting secret key").
What is the correct m i should be getting?
