Does this chat protocol exist?

Question

I want to implement a secure chat via mobile. I´ld like to know if the following protocol for key agreement – which can use any asymmetric encryption – exists.

Suppose Alice and Bob are two clients of the app. For this they have to register their public keys in a server:

1. Alice generates her public and private keys (pub_A, priv_A).
2. Alice sends pub_A to a public server.
3. Bob generates his public and private keys (pub_B, priv_B).
4. Bob sends pub_B to a public server.

Suppose A want to send a message m to B using AES:

1. The server generates a random symmetric key key_AES and encrypts it using pub_A and pub_B. Then it sends the encrypted key to Alice and Bob respectively.

Does this (chat) protocol for key exchange exist?

Answer 1

For this they have to register their public keys in a server:

1. Alice generates her public and private keys (pub_A, priv_A).
2. Alice sends pub_A to a public server.
3. Bob generates his public and private keys (pub_B, priv_B).
4. Bob sends pub_B to a public server.

So, you are basicly exchange public keys and the server uses those as encryption keys.

This means two things:

1. Anyone intercepting those public keys can forge messages, and
2. You are ignoring the fact why public and private keys come in pairs.

Practically, since your protocol does not use the private keys anyway, you might as well rename “public key” to “key” and notice yourself that you´re exchanging those keys (aka “secrets”) in public with no security what so ever. $User_{Key} \rightarrow Server$ is transmitted unprotected! Anyone eavesdropping the key exchange can intecept those keys and abuse them accordingly. Bob would have no chance to know if a received message was send and encrypted by Alice or Eve.

Also…

Suppose A want to send a message m to B using AES:

• The server generates a random symmetric key key_AES and encrypts it using pub_A and pub_B. Then it sends the encrypted key to Alice and Bob respectively.

Well, your protocol describes that you are expecting Alice and Bob to send their message in plain text to the server, and then the server encrypts the message to send it to Bob. What makes you think Eve will not abuse the fact you are sending “plain text” to the server? That marks another big flaw in your protocol: $User_{message} \rightarrow Server$ is transmitted unprotected!

Does this (chat) protocol for key exchange exist?

No. The obvious reason is that your idea of a secure protocol is absolutely insecure!

Besides that, your protocol idea is a prime example showing you why not to create your own crypto. There are ample well-vetted protocols available (for example diffie-hellman and ecc-based DH) which you can use as a base for mobile chat apps. Don´t shoot yourself in the foot by trying to invent something while better and more secure solutions already exist.

To get you going, I´d like to advise you to read up on both Diffie-Hellman key exchange and the (more recent) Elliptic curve Diffie–Hellman. Doing so will help you grasp how secure public/private key exchanges actually work, and (assuming you implement things correctly) each of them can be used to create secure chat apps.

Answer 2

I don't know of any chat protocol that would use this approach.

I get it from your question that once the symmetric key is established, Alice and Bob communicate directly using it and that it is not an issue that the server, if malicious, can defeat any security achieved by this.

However, it appears to me that your protocol suffers from a major problem, namely the fact that the server itself is not authenticated in any way. That means that an adversary with control of the network could impersonate the server, which by design knows the secret symmetric key Alice and Bob would be using.

Alice and Bob also never authenticate each other and rely on the lower layer and the supposingly secure shared key to communicate, so they may be impersonated very easily in such a scenario.