We are using the encryption built into Solaris 11 ZFS, which offers the choice between CCM (CBC counter mode) and GCM (Galois counter mode). What are the pros and cons of choosing each of these cipher modes?
TL;DR if you're reading this in 2020, applications should be using GCM mode.
CCM (Counter with CBC-MAC)
- Message authentication (via CBC-MAC) is done on the plaintext not the ciphertext. (This is generally not a desireable feature.)
- On the encrypt operation, the encryption and MAC could happen in parallel, but generally do not (typically because there is just one AES engine in a chip, just one AES thread at a time, etc.). Similar statement is true for decrypt.
- Performance costs essentially 2 x AES operations per block
- Cannot be parallelized
- CCM ciphers are available in OpenSSL as of TLS 1.3 (2018), but disabled by default.
GCM (Galois Counter Mode)
- GCM ciphers are the most widely used block ciphers worldwide. Mandatory as of TLS 1.2 (2008) and used by default by most clients.
- Message authentication (via GMAC/GHASH) is done on the ciphertext. (This is desirable most of the time.) Note that in most implementations, the auth check and decryption happen in parallel for performance reasons.
- Performance costs 1 x AES operation and 1 x GHASH per block (GHASH generally faster than AES, so GCM is faster)
- Encrypt/decrypt of multiple blocks can be parallelized nicely
GCM should be considered superior to CCM for most applications that require authenticated encryption. Because of the authentication that happens, GCM is not susceptible to the bit flipping and other attacks that can be mounted against counter mode or other stream modes.
There are some nuances that should be noted before using GCM that involve maximum size of the encrypted message and the MAC size. These are called out in the standard (NIST SP800-38D).