# Is this RSA signing and verification process secure?

I need to verify that messages in a communications program are from valid senders. Below is a highlevel outline of how I think the process would work. The connections themselves are done over TLS. The process here is just using RSA signing/verification on top of that. The data itself is actually not that important so the extra AES encryption is somewhat redundant. Can someone confirm whether this is the general approach or set me straight if not?

### Sending host

plaintext = "Hello World!"

#SIGN
digest = SHA256(plaintext)
sig = rsa_encrypt(sender_priv_key,digest)

#ENCRYPT DATA
iv = generate iv
key = generate random key
ciphertext = aes_encrypt(iv,key,plaintext)

#ENCRYPT KEY
encryptedKey = rsa_encrypt(recipient_pub_key,key)

#SEND
send(sig+iv+encryptedKey+ciphertext)


### Receiving host

#RECEIVE
sig+iv+encryptedKey+ciphertext = recv()

#DECRYPT KEY
key = rsa_decrypt(recipient_priv_key,encryptedKey)

#DECRYPT DATA
plaintext = aes_decrypt(iv,key,ciphertext)

#VERIFY
sigDigest = rsa_decrypt(sender_pub_key,sig)
msgDigest = SHA256(plaintext)

if sigDigest == msgDigest
#valid sender so long as we can trust where we got the public key
#data integrity is good

• sig = rsa_encrypt(sender_priv_key,digest) has at least a terminology problem (in public-key crypto, the private key is used to sign or decipher, not encipher); and very possibly a security problem if what is meant is: digest raised to private exponent modulo public modulus, which is not a proper way to sign. – fgrieu Jul 17 '16 at 16:00
• @fgrieu Thanks for you comment. Isn't the process of signing the message just encryption of some value using the private key. The sender is the only one with knowledge of the private key (i.e. private exponent). Decrypting with the public key (what I understand to be verifying) reverses the work done by the first part. If not encrypting with the private key how is the message signed using the private key? what is the process? – Steve Jul 17 '16 at 19:52
• – dave_thompson_085 Jul 18 '16 at 2:52
• ... although some software (IME the oldest moldiest 'eay' part of OpenSSL, and part of the Java crypto API -- shame on you Sun) do confuse matters by miscalling it that. – dave_thompson_085 Jul 18 '16 at 2:57
• Is there any reason why you don't want to use CMS or OpenPGP formats? – SEJPM Jul 18 '16 at 5:43

Comments already pointed out, that encryption and signing are not the same and should not be exchanged deliberately. Of course in practice specifically for RSA, not PKE in general, and only in the textbook variant (no padding), encrypt/decrypt are bascally the same operations as sign/verify: For all of them, you just do modular exponentiations; and the defining properties of the keys (the product of the public and private exponents equals $1$ modulo $\phi(N)$) are analogously.

But considering your algorithm, there are flaws. First, your algorithm for signing is actually this:

• Hash the message
• "rsa_encrypt"the hash with a private key

First, if we interpret "rsa_encrypt" just as modular exponentiation, this is still just textbook-RSA without any proper padding , which is vulnerable to certain attacks and usually the padding scheme PSS is recommended. (also see wiki)

And then you have the implicit assumption, that your function call actually is the plain modular exponentiation: If you call "rsa_encrypt" from a cryptographic library, are you actually sure that it is just the modular exponentiation? It could e.g. already have soem kind of padding in there, which might be unsuitable for signatures. Or did you implement the function yourself? Because then of course there is the old saying "don't implement crypto yourself" (and the question doesn't sound like homework), and for good reasons any proper cryptographic library will offer separate methods for signatures and encryption. In the implementation details, there are quite a lot more differences when it comes to side-channels, performance improvements, etc.

I would recommend, try to get rid of your current primitives, and use a cryptographic library instead, which gives you proper signature algorithms. Other than that, you are e.g. generating the random key for the symmetric part of the hybrid encryption. You shouldn't have to do that manually, because it is another possible weakness (you have to ensure that you're using a proper CSPRNG). But most cryptographic libraries should also have that functionality.

Your approach is correct. I will just point out that you get a bit more than integrity.

Because you are using a signature you also get:

• authenticity: the data are from sender A and not sender B
• non repudiation: sender A cannot deny he sent this data.