# Detect MITM with hashed session-key — Is my protocol secure?

I have the following setup:
An app ("client") wants to communicate securely with a device I created ("server").
The client has the public key of the server (and a hashed password) preinstalled.

I use the libhydrogen crypto library (inspired by libsodium / same creator as libsodium).

As far as I know, I'm still vulnerable to MITM attacks because the key exchange does not protect me from that.
To check if there is a MITM I would like to compare if both parties have the same symmetric session key.

Q1: If I am able to compare that the session keys and they are the same, can I assume that the connection is secure (no MITM present)?

In order to do that comparison, my idea is to send a packet containing a hash of the session key + a hashed password from the client to the server. This packet will be encrypted using the public key of the server.

This means that only the server is able to decripty the message and will be able to verify that they use the same session key (+ I can use that to authorize the client). Because of the password, a MITM will not be able to create the same message with his false session key.

Q2: Are my assumptions correct or did I miss something?

Q3: Could I do the whole thing simpler or am I on the right track? (If I do not want to use TLS)

If I am able to compare that the session keys and they are the same, can I assume that the connection is secure (no MITM present)?

Yes, baring attack on the extremities. That's a property of the Diffie-Hellman key exchange.

However, it is not satisfactory that the client sends (as envisioned) $$\text{ENC}_\text{PK}(\text{Hash}(\text{Sym.Session Key})\mathbin\|\text{Hash}(\text{Password}))$$ (without using further encryption with the session key) in order to check that the two ends share the same session key. The security depends on details of the public key encryption used. For example, in some forms of RSA encryption, we would have what's sent equal to $$\text{ENC}_\text{PK}(\text{Hash}(\text{Sym.Session Key}))\mathbin\|\text{ENC}_\text{PK}(\text{Hash}(\text{Password}))$$, the right term could be extracted by mere passive eavesdropping, then allowing that attacker to authenticate w.r.t. the server.

Also, there's the issue that libhydrogen does not seem to offer public key encryption (at least glancing at the table of content).

The simplest working solution is to send $$\text{Hash}(\text{Password})$$ within the secret-key authenticated encryption channel, as the first payload, and have the server decipher and check that against the correct value.

A convincing security argument (not proof) that the above is enough for security is that

• In a MitM attack modifying what's exchanged before the packet carrying the above payload, an attacker can't impersonate the server w.r.t. the client. The client will abort before anything related to $$\text{Hash}(\text{Password})$$ is sent.
• In an attack (passive or active) starting after that, the attacker has no knowledge of the session key, and that key won't be reused in another session. Hence whatever the adversary intercepts that depends on $$\text{Hash}(\text{Password})$$ does not leak information about that; and (independently) is of no use in later sessions.
• The use of correct authenticated encryption should take care of ensuring that whatever the adversary intercepts that depends on $$\text{Hash}(\text{Password})$$ is of no use in the current session (in particular, it is necessary that individually authenticated packets can't be replayed; if they could, replay of the packet with $$\text{Hash}(\text{Password})$$ in the right context could leak information about its plaintext).

However, the above simplest working solution is a tad brittle: if the server's private key leaks, an attacker impersonating the server w.r.t. a client can obtain the hash of the password (which is as good as the password for authentication purposes). And if a session keys leaks, passive eavesdropping allows the same.

Rather (and following the reasoning in the first part), we could send some MAC with key $$\text{Hash}(\text{Password})$$ and message $$\text{Sym.Session Key}$$, which the server can check. HMAC-SHA-256 would be suitable. There's an important incentive to further send that MAC thru the secret-key authenticated encryption channel: it prevents an adversary from checking a guess of the password unless s/he has the server's private key.

Note: If password is to be short enough to be conveniently keyed-in, or otherwise low-entropy, then $$\text{Hash}(\text{Password})$$ should still preferably be with a slow hash intended for passwords.

Note: No matter what, the password should have enough entropy to resist online password guesses, where an adversary attempts to authenticate w.r.t. the server using a dictionary of passwords. Limitation of the number of invalid attempts on the server (perhaps by a mere delay) can help with this.

• Thanks for your answer! What's not clear to me is how your "simplest working solution" would work. Why is the MITM not able to decrypt the messages? If he sits between the client and server and has a Secret-key authenticated encryption connection with both of them, won't he just decrypt and re-encrypt every message sent between the two and thus be able to read hash(password) and forward it? – Thomas Nov 14 at 14:29
• Your 2nd solution is clever — i don't think I'm able to implement this with libhydrogen - but that's another problem... – Thomas Nov 14 at 14:31
• @Thomas: the correctness of my simplest working solution is indeed far from evident! I have made my best to justify it. Regarding lack of MAC: building HMAC from common hashes (SHA-2) is very easy and quite foolproof. And it's unnecessary with SHA-3, which is even easier to turn into a safe MAC. – fgrieu Nov 14 at 15:57