1
$\begingroup$

Encryption process for any outgoing packet to the specified "remote IP":

-original payload P is encrypted (the bytes after the IP header) using RC4 with the preestablished key KEY to obtain encrypted payload C=RC4(P, KEY)

-The encrypted payload C is concatenated with the 16 byte output of MD5(C|KEY), and the IP header is adjusted accordingly (increasing the packet total length, re-calculate the checksum) so that the outgoing packet will have payload C | MD5(C|KEY) and destination "remote IP"

Decryption process for any incoming packet from specified "remote IP":

-Check the packet payload for the MD5 authentication with the same given "key" Specifically, divide the received packet payload (those bytes after the IP header) as two parts C, A where A is the last 16 bytes of the packet payload and C is the rest.

-If MD5(C|KEY) == A, decrypt C using RC4 to obtain P=RC4(C, KEY), change the payload of the incoming packet from C|A to P, and adjust the IP header accordingly (decrease the packet total length, re-calculate the checksum) so that the incoming packet will be restored to its original form without keyed md5 authentication before sent to the receiving process

Improve this question
$\endgroup$
8
  • $\begingroup$ Is the RC4 key the same as the key passed to MD5? $\endgroup$
    – Adrian Self
    Commented Apr 25, 2020 at 20:41
  • $\begingroup$ Does attacker know whether the md5 authentication was accepted? $\endgroup$
    – Adrian Self
    Commented Apr 25, 2020 at 20:42
  • $\begingroup$ How is the key shared between the parties? $\endgroup$
    – Adrian Self
    Commented Apr 25, 2020 at 20:44
  • $\begingroup$ Thanks for the questions and your response. Yes the rc4 key is the same as the key passed to md5. If authentication has failed, an error message is shown and the incoming packet is dropped. The key is shared prior to the communication and both of them have the same key. $\endgroup$
    – Koolz
    Commented Apr 25, 2020 at 20:48
  • $\begingroup$ "If authentication has failed, an error message is shown" to the sender of the packet? $\endgroup$
    – Adrian Self
    Commented Apr 25, 2020 at 20:52

1 Answer 1

Reset to default
1
$\begingroup$

1) Are replay attacks possible?
Yes. For example, if Alice wants to tell Bob "Pay Eve \$100", then
P = "Pay Eve $100"
C = RC4(P, KEY)
Packet = IP(C | MD5(C|KEY) )
By capturing and replaying Packet, Eve can make Bob pay her \$100 over and over again.


2) Are Man-In-The-Middle attacks possible?
More specifically, "Is there a possibility to intercept messages and impersonate at-least one of the parties? Is it possible for the attacker to put himself between the sender and receiver by breaking the encryption?"

This question is more complicated. In short, it depends.

If the attacker gets hold of plaintext / ciphertext pairs for the RC4 (either through encryption oracle or decryption oracle), he simply takes P xor C to find the keystream. RC4 key reconstruction attacks can be used to find the MD5 key. The attacker may then act as a full MITM, intercepting and viewing messages and optionally modifying them before passing them on.

I am also uncomfortable with that MAC construction, so I'll get back to you on that one.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for the response, Can you please explain a bit more on how or why a replay is possible? $\endgroup$
    – Koolz
    Commented Apr 25, 2020 at 21:09
  • $\begingroup$ More explanation added. $\endgroup$
    – Adrian Self
    Commented Apr 25, 2020 at 21:17
  • $\begingroup$ Appreciate the explanation! Any idea about Man in the middle attack in this scenario? $\endgroup$
    – Koolz
    Commented Apr 25, 2020 at 21:21
  • $\begingroup$ By MITM attack, do you mean modifying the message in transit yet having the receiver accept the message integrity? $\endgroup$
    – Adrian Self
    Commented Apr 25, 2020 at 21:26
  • $\begingroup$ Is there a possibility to intercept messages and impersonate at-least one of the parties? Is it possible for the attacker to put himself between the sender and receiver by breaking the encryption ? $\endgroup$
    – Koolz
    Commented Apr 25, 2020 at 21:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.