I have to create a Client-Server Application using Java and I want to make the communication secure. I thought to use AES to encrypt the messages and for key exchange I make the next steps:

  • Client generates RSA keys and send public key to server
  • Server will encrypt the AES key with RSA Public key (AES key is generated random for every client)
  • Client decrypt the message with its RSA Private key and then all messages will be encrypted with AES

My question is: is it a good practice ?

  • 3
    $\begingroup$ Basically, what you've described is key wrapping using a public/private key. It can also be done with symmetric keys as well. Rather than invent your own way of doing it and possibly making a mistake you should take an existing implementation from a good crypto library and use that. $\endgroup$ Dec 23 '21 at 14:20
  • 3
    $\begingroup$ The real question is: how are you going to trust the public key that you receive? It might be one created by an adversary. This is usually where PKI comes into play - quite often PKIX with X.509 certificates as used in e.g. your browser for TLS (server) authentication. $\endgroup$
    – Maarten Bodewes
    Dec 23 '21 at 22:13
  • 1
    $\begingroup$ Otherwise yes, RSA can be used to perform key establishment, with the main drawback that key pair generation is pretty slow - so if you want to use a new key pair for each connect for forward security, then you may have an efficiency problem. This is why commonly (EC)DH is used instead. $\endgroup$
    – Maarten Bodewes
    Dec 23 '21 at 22:15
  • 2
    $\begingroup$ Does this answer your question? Is direct RSA encryption of AES keys secure? $\endgroup$
    – kelalaka
    Dec 29 '21 at 22:55

No, it is not a good practice.

  1. It is prone to man in the middle attack. When server receives a public key, it does not know if this key comes from a client or from the "man in the middle". Thus, the "man in the middle" can intercept the traffic between client and server, for client it can simulate server, for server it can simulate client. Thus, the whole communication will be 1) known to MitM and 2) data sent in both directions can be modified by MitM.
  • --> You can avoid it, if you authenticate clients to server or authenticate server to clients by using public key certificates.
  1. It is prone to replay attack. Suppose a client sends command "transfer 1000 USD to account 123456789". This can be legitimate operation. But if an attacker intercepts the traffic and resends it within relatively short time (so that the server still uses the same key for AES encryption for this client), then server will not be able to distinguish, if this traffic comes from the client of from the attacker, and will execute the command. Even if you authenticate clients to server or authenticate server to clients, this problem will still persist.
  • --> You can avoid it, if you use nonce.
  1. The good thing: Your current approach has relatively good forward secrecy: past sessions are protected against future compromises of client keys or session passwords. But if you address the problem 1 mentioned above (authentication) and start using the same RSA key over multiple sessions, then you will lose forward secrecy. If an attacker obtains private key, all past sessions can be decrypted. To prevent it, you will need further measures, e.g. ephemeral Diffie-Hellman key exchange.

  2. Independent on all the problems above, it is important to properly use RSA, as @kelalaka mentioned. See details here and here.

Thus, it may be much more secure to use TLS implementation provided by Java, instead of implementing your own protocol.

  • $\begingroup$ Maybe you can add: for TLS 1.3, the standardization committee threw out RSA for key agreement completely. What the history around PKCS#1 and Bleichenbacher attacks has shown: RSA might just be a bad choice for key exchange, and it's not worth the hassle to get it right, when all problems are so much easier to fix when using other methods $\endgroup$
    – tylo
    Jan 6 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.