Skip to main content

If two hosts are using a transparent IP level authentication and encryption, is their communication subject to both MITM or replay attacks?

Encryption process for any outgoing packet to the specified "remote IP":

-original payload P is encrypted (the bytes after the IP header) using RC4 with the preestablished key KEY to obtain encrypted payload C=RC4(P, KEY)

-The encrypted payload C is concatenated with the 16 byte output of MD5(C|KEY), and the IP header is adjusted accordingly (increasing the packet total length, re-calculate the checksum) so that the outgoing packet will have payload C | MD5(C|KEY) and destination "remote IP"

Decryption process for any incoming packet from specified "remote IP":

-Check the packet payload for the MD5 authentication with the same given "key" Specifically, divide the received packet payload (those bytes after the IP header) as two parts C, A where A is the last 16 bytes of the packet payload and C is the rest.

-If MD5(C|KEY) == A, decrypt C using RC4 to obtain P=RC4(C, KEY), change the payload of the incoming packet from C|A to P, and adjust the IP header accordingly (decrease the packet total length, re-calculate the checksum) so that the incoming packet will be restored to its original form without keyed md5 authentication before sent to the receiving process

Koolz
  • 13
  • 4