how does AES GCM in SSH work?

Question

I was looking at RFC5647: AES Galois Counter Mode for the Secure Shell Transport Layer Protocol and had some questions about section 7.1 "IV and Counter Management". Here's that section:

7.  Processing Binary Packets in AES-GCM Secure Shell

7.1.  IV and Counter Management

   With AES-GCM, the 12-octet IV is broken into two fields: a 4-octet
   fixed field and an 8-octet invocation counter field.  The invocation
   field is treated as a 64-bit integer and is incremented after each
   invocation of AES-GCM to process a binary packet.

         uint32  fixed;                  // 4 octets
         uint64  invocation_counter;     // 8 octets

           Figure 3: Structure of an SSH AES-GCM Nonce

   AES-GCM produces a keystream in blocks of 16-octets that is used to
   encrypt the plaintext.  This keystream is produced by encrypting the
   following 16-octet data structure:

         uint32  fixed;                  // 4 octets
         uint64  invocation_counter;     // 8 octets
         uint32  block_counter;          // 4 octets

           Figure 4: Structure of an AES Input for SSH AES-GCM

   The block_counter is initially set to one (1) and incremented as each
   block of key is produced.

   The reader is reminded that SSH requires that the data to be
   encrypted MUST be padded out to a multiple of the block size
   (16-octets for AES-GCM).

So let's say you invoke AES GCM twice and that the invocation counter is 0 (for simplicity sake). The first invocation encrypts two blocks and the second invocation encrypts one. What would be the source material for the encrypted keystream the second invocation?

For the first invocation, first block, it'd be this:

     uint32  fixed;
     uint64  0;
     uint32  1;

For the first invocation, second block, it'd be this:

     uint32  fixed;
     uint64  0;
     uint32  2;

For the second invocation, first (only) block, does the block_counter advance, as well, or does it reset? Here's what it'd be in the case of the former:

     uint32  fixed;
     uint64  1;
     uint32  3;

Here's what it'd be in the case of the latter:

     uint32  fixed;
     uint64  1;
     uint32  0;

eg. is the block_counter reset or incremented when the invocation_counter incremented?

Also, unless I'm mistaken, this approach breaks with tradition for SSH encryption in that encrypt(S1) || encrypt(S2) == encrypt(S1 || S2) (where || is the concatenation operation). That's how pretty much how it works with every other symmetric cipher for SSH but not this one.

Answer 1

No, the block counter is reset, because you will reinitialize the GCM mode with a new IV/nonce.

The scheme you mentioned with encrypt(S1) || encrypt(S2) == encrypt(S1 || S2) is broken anyway when it comes to GCM due to the authentication tag. Besides that, you'd always want to verify the decryption before using the plaintext anyway, so decryption should be one (partial) message at the time. If you don't then GCM is basically useless.

---

However, the first counter value does not start at one, but it starts at value two. This is because the initial CTR block is generated with counter 1 instead of 0. This block is however used to calculate the $H$ component required to perform the GMAC authentication. So the ciphertext itself is encrypted starting at block one. You can see an application of it here (my answer on SO) where I decrypt GCM ciphertext while skipping verification.

It probably doesn't start with value 0 because retrieving a Key Check Value is usually performed by encrypting a block of zeros, i.e. counter value 0 with an all zero nonce. Starting at zero would compromise the calculation of H and thus GMAC and the resulting authentication tag. But that's just speculation.

Answer 2

The CTR-IV either must be random or regularly incremented in a way that the IV never repeats again. If there is a repetition then the encryption will be insecure.

In rfc5647, it is the second way; regularly incremented.

• invocation_counter is incremented per package. Section 7.1;

  The invocation field is treated as a 64-bit integer and is incremented after each invocation of AES-GCM to process a binary packet.

• block_counter is incremented per AES encryption within the package. It must be initialized to 1 for per package, otherwise, an overflow will occur and therefore the IV's cannot be unique. Section 7.1;

  The block_counter is initially set to one (1) and incremented as each block of key is produced.

The 4-octet block counter matches with the size of the package that is defined in section 5.2 as;

uint32    packet_length;  // 0 <= packet_length < 2^32

Thus, this counter can be incremented for all encrypted blocks within one package. For a new package the set to 1 (that is your latter case).

For the second part, this can happen if ECB mode is used and the data sizes are multiple of the block cipher. Even in this case, a padding scheme must be applied. that will prevent your equation.