I was configuring a VPN on a firewall (fortigate) and realized that I could use AES-GCM for encryption in the IKEv1 phase 2 but not in the phase 1 and I was wondering why knowing that in IKEv2 we could use it in both phases. I have already checked the RFC articles but there was no explanation. Is it only because AES-GCM came after IKEv1 or something like that?

  • $\begingroup$ I don't know, but the Wikipedia sentence "Fewer cryptographic mechanisms: IKEv2 uses cryptographic mechanisms to protect its packets that are very similar to what IPsec Encapsulating Security Payload (ESP) uses to protect the IPsec packets. This led to simpler implementations and certifications for Common Criteria and FIPS 140-2, which require each cryptographic implementation to be separately validated." strongly suggests this; IKEv1 is old - almost prehistoric in cryptographic sense. Using a strong cipher mode in an insecure/badly specified protocol doesn't make much sense. $\endgroup$ – Maarten Bodewes Sep 19 '19 at 10:51
  • 1
    $\begingroup$ It feels weird that it would be accepted for phase 2 but not phase 1. Is there a difference between both phases that would make the use of gcm acceptable in phase 2 ? $\endgroup$ – user1990088 Sep 19 '19 at 11:20
  • $\begingroup$ Sure, and although I could think of a few reasons from a protocol design point of view, I don't know the answer to that, so I also upvoted your question :) $\endgroup$ – Maarten Bodewes Sep 19 '19 at 11:21
  • $\begingroup$ Probably. IKEv1 was published in 1998. GCM seems to have been finalized around 2003. Not that it couldn't have been retroactively added, but IKEv2 came out in 2005 so why work on improving the security in an old protocol rather than in the new one that also addressed other issues? $\endgroup$ – Swashbuckler Sep 19 '19 at 14:22

With IKEv2 all payloads are generally sent in a single Encrypted Payload (SK) whose data is both encrypted and integrity protected/authentication (that protection covers the complete message, including the header and optional unencrypted payloads). That works quite nicely with modern AEAD algorithms like AES-GCM, the additional authenticated data (AAD) are the headers (IKE and SK) and any unencrypted payloads, the resulting authentication tag is what's sent as Integrity Checksum Data in the Encrypted payload.

On the other hand, IKEv1/ISAKMP handles encryption and integrity protection of IKE messages quite differently. For integrity protection of Phase 2 messages (Quick Mode/Informational), the message ID and the payloads are hashed (a PRF is used actually) and the result is added to the message in an additional Hash payload (during Main and Aggressive Mode it's similar but slightly differs based on the authentication method). The complete message, including that Hash payload, is then encrypted. So that does not really match with how AES-GCM works, unless the protocol would have been changed significantly (like getting rid of the redundant Hash payloads and handling integrity protection only via combined-mode cipher). But I guess there really was no point in doing that for an already obsolete protocol.

After writing this answer, I noticed that this is actually described in the introduction to RFC 5282.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.