Signature generation using hash function

I want to generate a valid signature for a message using the below algorithms:

1. Hash function: $$H(x) = (x+3) \bmod 10$$ (I am unable to understand the given parameter for hash function and how am I supposed to use it in hashing a message.)

2. Encryption: RSA with the following parameters

Private key: 11

Public key: 5

Modulus: 14

(I had tried but failed in using these parameters in RSA encryption. I want to know the possible way or hint to be able to use these exponents in encrypting the hash of a message.)

• Welcome to Cryptography. This is obviously homework question that we only provide some hint if you show some effort as our current policy. So with your knowledge what have you tried and wher you stuck? You can edit your anser. Aug 21 '20 at 19:12
• Also, RSA signatures and RSA encryption are different: they use different padding to achieve different security goals. Conflating them is like saying that bicycles and pickup trucks are the same thing, since both have wheels. Aug 21 '20 at 19:23
• @SAIPeregrinus there is also Full Domain Hash Aug 21 '20 at 20:14
• Yes, and that's still no closer to encryption than any other signature scheme. RSA-KEM is sort of encryption, but you can't pick the value freely (unlike with OAEP). Accumulators are another very different use of the same mathematical operations. Aug 22 '20 at 23:30

The private key is actually the private exponent. Signature generation is modular exponentiation using the private exponent and - of course - the modulus. So you first hash a value for x using the hash function, giving you another value, say $$h$$. Now you perform the modular exponentiation on it and return the result. And that's all there is to it.