Hot answers tagged challenge-response
Yes, there are several ways in which Mallory could pretend to be Amy. One obvious way, which doesn't even involve Amy herself in any way, would be for Mallory to perform steps 1 and 2 of the protocol normally, as if he were Amy. Then, given Betty's nonce $n_b$, Mallory can start a second, parallel instance of the protocol, again pretending to be Amy, and ...
This is vulnerable to a length extension attack. Given a valid nonce/MAC, the nonce can be extended to forge a new valid nonce/MAC value. This is because $m_4$ is appended to the end inside the outer hash. How this affects you will depend on how you validate your nonce. But in general, this is not a secure construction. There's probably more things wrong ...
The Wikipedia article points out a good reason for using a random challenge value: preventing replay attacks. If the hash was always the same (as the hash of the symmetric key would be), then having listened in on one challenge-response cycle, a malicious listener could pass further handshake tests.
Yes, because Mallory can use Amy and Betty to get any encrypted nonce; Amy and Betty are oracles for Mallory. She just has to send the nonce she has to encrypt to either one of them and they perform the task for her (in another "authentication attempt", using step 1 & 2). Usually you protect against this kind of situation by performing an encryption ...
Only top voted, non community-wiki answers of a minimum length are eligible