I am trying to validate my RSA implementation for digital signatures with the test vectors provided by NIST. For now, I am focusing on the tests for signature generation (SigGen15) with the PKCS#1 v.15 padding scheme.
Given a message M, the modulus N and the private key D, I need to find the signature S of M. From my understanding, first I need to calculate the hash H of the message N and then create the following sequence of bits:
0x00 | 0x01 | P | 0x00 | S | H
where P is a sequence of OxFF so that this sequence of bits has the same bit length of N, S is the sequence of magic bytes (e.g., for SHA-1 is 0x30 0x21 0x30 0x09 0x06 0x05 0x2b 0x0e 0x03 0x02 0x1a 0x05 0x00 0x04 0x14) and H is the hash of the message M. Then, I can encrypt this sequence with the private key D and the modulus N (power + mod).
Unfortunately, I can't understand how I am supposed to generate the hash H. By using the NIST-generated signature, I can see that the hash code H expected in the message is different from the one I can generate by simply calculating H=sha1(M). Everything else, i.e., the number of bytes in P and the magic code S, is correct. It seems that the message M is manipulated somehow before calculating the corresponding hash code H to be used for the generation of the signature. I checked the documentation but I can't figure out what I am doing wrong.
Thank you so much.