2
$\begingroup$

I am implementing a custom challenge-response protocol to authenticate a client to a server. Mutual authentication is not required, i.e. the server doesn't need to be authenticated to the client.

The protocol works over TCP. Client and server are in a LAN (not over the internet). There is only one server and only one client. The authentication is designed as follows:

$S \rightarrow C : N$

$C \rightarrow S : \{N\}_K$

Where the server $S$ creates a 128-bit random number and sends it as nonce $N$ to the client $C$. The client encrypts the nonce with AES-192 with the pre-shared key $K$ and sends it as response to the server. Then the server decrypts the message and checks the nonce.

Questions:

  • Is this protocol design secure?
  • What would be the advantage to use HMAC (or any other MAC) instead of AES encryption?
  • Did I miss something?
Improve this question
$\endgroup$
1
  • $\begingroup$ What exactly you beam be secure? $\endgroup$
    – mentallurg
    Commented Feb 24, 2020 at 21:55

2 Answers 2

Reset to default
2
$\begingroup$

Is this protocol design secure?

See below.

What would be the advantage to use HMAC (or any other MAC) instead of AES encryption?

HMAC was originally proposed as a construct that turns a Merkle-Damgaard hash function based on compression functions built from block ciphers, into a message authentication code. Although there's no decryption in HMAC, you can actually easily replace your AES decryption-based authentication with comparing HMAC-signed tags.

Did I miss something?

What you're doing is essentially an authenticated security transport without confidentiality.

What you're missing is that you didn't authenticate the exchange following the initial authentication - this allows for connection hijacking, arbitrary injection of data packets, and more.

Therefore, you should use HMAC+[hash] or CMAC+[block cipher] to authenticate all of your messages exchanged, and also keep a counter as a state to detect duplicate packets and message replay attackes.

Improve this answer
$\endgroup$
-1
$\begingroup$

Is this protocol design secure?

This design says that server sends a nonce which the client gladly accepts. How does the client knows that nonce is coming from the expected server? There is no integrity check performed for the nonce sent by server.

Encryption will only provide the confidentiality. It would be desirable if the data is also added with MAC (such as CMAC) for integrity; this should complete the design and make it more robust.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Commonly this kind of exchange within a challenge response protocol doesn't require integrity check. If you want to argue for it then you'd have to include a reason why it needs to be performed. Note that it even can introduce vulnerabilities if you both encrypt and perform a MAC within a challenge response protocol in weird but feasible circumstances. $\endgroup$
    – Maarten Bodewes ♦
    Commented Feb 27, 2020 at 11:20
  • $\begingroup$ thanks for the comment. Can you please elaborate on the vulnerabilities which can crop up if we both encrypt and perform a MAC within a challenge response protocol. $\endgroup$
    – Vikash
    Commented Feb 28, 2020 at 6:53
  • $\begingroup$ No, because that would indicate too much of what I was doing at the time :) If I'd thought I could include it I would have. $\endgroup$
    – Maarten Bodewes ♦
    Commented Feb 28, 2020 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.