What are the rules for using AES-GCM correctly?

Question

When using AES-GCM I know that I am supposed to use a new initialization vector every time I call the AES-GCM algorithm with the same key. What are other rules must be followed to use AES-GCM correctly?

I am looking for a bullet point checklist with advice that I can follow as a developer, and the consequence of ignoring the advice.

I am using the Java implementation of AES-GCM.

Answer 1

Rules or your obligations

1. Key generation: select a key uniform randomly, and keep it secret.

2. (Key,IV) Resue: An $(IV,Key)$ pair must never be used again.

3. Maximum file size: The recommended IV is 96 bits so 32 bit counter left. This means that the counter can count at most $2^{32}$. the first two are used so there is at most $2^{32}-2$ calls of CTR encryption. Therefore for an IV, one can encryption at most $2^{32} - 2$ blocks, or $2^{36} - 32$ bytes, or $2^{39} - 256$ bits. This is around $\approx 68$ gigabytes. You should stop way before, see more in advices section.

4. Check the authentication tag before decryption. If there is a mismatch, throw an error.

5. Tag size: use the recommended tag size 128-bit. You should not trim the tag size too much. For the full details of the tag security, see this nice posts;

   • Does GCM (or GHASH) only provide 64-bit security against forgeries?

   • GMAC vs HMAC in message forgery and bandwidth

6. Key size: Prefer 256 bit, though 128 bit AES is secure. In AES one will slow around %40 if uses 256 bit AES instead of 128-bit AES.

7. IV size: One can provide an IV longer than 96 bits, however, that will cause an additional GHASH call to convert into 96-bits. $$\textbf{if } \operatorname{len}(IV) = 96 \textbf{ then } Y_0 = IV \mathbin\| 0^{31}1 \textbf{ else } Y_0 = \operatorname{GHASH}(H, \{\}, IV)$$

   With larger One lost the counter/LFSR's deterministic IV generation. Stick to use 96 bits.

---

Advices

1. For IV generation, NIST suggests two types 800-38D

   1. Deterministic Construction In a short manner use counter/LFSR to prevent IV reuse under the same key. One needs to store the last counter/stage to advance. During system failures, these values may not be recovered correctly, A combined mode half random, half counter/LFSR can eliminate this or start with a fresh key.

      This counter should not be confused with the internal counter used in the CTR encryption.

      The deterministic IV generation with an LFSR recommendation of the NIST is very effective in the case of a hardware implementation of the AES-GCM as pointed in this answer

   2. RBG-based Construction RGB is a Random Bit Generator. In short from the NIST: The random field shall either consist of

      1. an output string of r(i) bits from an approved RBG with a sufficient security strength, or
      2. the result of applying the $r(i)$–bit incrementing function to the random field of the preceding IV for the given key

2. Random IV vs Sequential IV

   AES-GCM best works 96 bit IV and when the IV is generated sequentially. If you choose random IV then due to the IV collisions under the same key, then the limit on the number of messages to encrypt under the same key is much smaller than sequential 96-bit IV.

3. Limit the message size gainst forgery: Put a limit to message size to eliminate the adversary's forgery advantage. It is linear in the maximum message size (not, e.g., the average): merely sending a single $2^{36}$-byte message makes the adversary's task four million times easier than if you limit all your messages to $2^{14}$

4. Limit the message size gainst DOS attack: Put a limit to message size to prevent Denial-of-Service by selecting the amount as the size of memory can an adversary reject before rejecting for forgery.

5. If you need to use AES-128; during the negotiation for the key agreement, it is better if the pair not only agrees on a key but also on a random initial counter. Then increment on it with a counter/LFSR.

---

Little deeper

1. (IV,key) reuse: This can lead to lost of confidentiality since the underlying encryption is performed by the CTR mode.

   A CTR mode converts a block cipher into a stream cipher. Like all stream ciphers, if the keystream is reused crib-dragging technique can reveal the messages. Once the messages are removed the keystream is also revealed. However, this doesn't mean that AES is broken here. One still needs to execute a KPA attack and AES has known to be secure.

   Reuse can also have catastrophic result of forgery.

---

Thanks to SqueamishOssifrage's comments and suggestions on the chat.