How RSA is implemented by authentication protocols?

Question

I am a beginner in cryptography, i recently learned about RSA algorithm : In short, my understanding for this is : Sender needs receiver's public key to encrypt the message to a cipher so that only receiver can decrypt the cipher text to original message and vice versa.

When i try to understand the implementation in network and authentication i find this little confusing in following ways:

1. Sender and receiver both generate their own pair of private-public key.
2. Sender encrypts the message with private key and sends his/her public key and the receiver decrypts by using sender's public key and vice versa.
3. Now how this is RSA or AES? Because the message should be encrypted by receiver's public key so that receiver can only decrypt it. But here sender's public key is sent and receiver could decrypt the message by that public key. HOW??
4. If this is true, then any one in the network who can get cipher and public key can decrypt the message, because the cipher is meant to be decrypted by the same public key. RIGHT??

For example : I was going through JWT (Json Web Token) implementation, here :

• Server app signs the token with server's private key but third party app can verify the token and retrieve data by the server's public key.

I know i am missing something here, but do not know what.

Answer 1

Sender encrypts the message with private key and sends his/her public key and the receiver decrypts by using sender's public key and vice versa.

No, you cannot encrypt (for confidentiality) using a private key; anybody with the public key can decrypt after all.

Now how this is RSA or AES? Because the message should be encrypted by receiver's public key so that receiver can only decrypt it.

What you are looking for is signature generation, which is different from encryption. For RSA it uses a cryptographic hash and different padding mechanisms. The "only" thing that is similar is that it uses modular exponentiation. Both encryption and signature generation depends on the RSA problem for security, and the RSA primitive is nothing but modular exponentiation (after a rather complex key pair generation).

The unfortunate thing is that because encryption and signature generation is so similar, the authors of PKCS#1 called it "encryption of the hash value using the private key". They somewhat rectified that mistake in v2.0 of their specification, but then it was too late.

AES doesn't take any part in signature generation. It is mainly used for encryption, including hybrid crypto-systems where RSA and AES are often combined to encrypt larger messages. It is not used or designed for hashing.

But here sender's public key is sent and receiver could decrypt the message by that public key.

Well, if you perform modular exponentiation with the public key then you get the value that was input to the modular exponentiation with the private key. This contains the hash value (in some form). This is the base principle behind RSA signature generation and verification - the padding is just there to provide additional protection against attacks.

There is another misunderstanding here: the public key cannot just be "send", it must be trusted by the party that is doing the verification. Otherwise you may get send the wrong key, and you would perform verification of a signature that could be generated by anybody.

Establishing trust in a public key is the reason behind public key infrastructures. The certificate store in your browser is part of a PKI called PKIX as it relies on X.509 encoded certificates and the technology surrounding it. PGP's "web-of-trust" is another, less hierarchical PKI.

If this is true, then any one in the network who can get cipher and public key can decrypt the message, because the cipher is meant to be decrypted by the same public key.

Correct, but we don't care. Because keeping the message confidential is not the goal. It is about proving that you have access to the private key so you can sign some session specific data. It needs to be session specific, otherwise an adversary can use an old signature to authenticate (a so called replay attack).