We are using the encryption built into Solaris 11 ZFS, which offers the choice between CCM (CBC counter mode) and GCM (Galois counter mode). What are the pros and cons of choosing each of these cipher modes?
TL;DR if you're reading this in 2020, applications should be using GCM mode.
CCM (Counter with CBC-MAC)
- Message authentication (via CBC-MAC) is done on the plaintext not the ciphertext. (This is generally not a desireable feature.)
- On the encrypt operation, the encryption and MAC could happen in parallel, but generally do not (typically because there is just one AES engine in a chip, just one AES thread at a time, etc.). Similar statement is true for decrypt.
- Performance costs essentially 2 x AES operations per block
- Cannot be parallelized
- CCM ciphers are available in OpenSSL as of TLS 1.3 (2018), but disabled by default.
GCM (Galois Counter Mode)
- GCM ciphers are the most widely used block ciphers worldwide. Mandatory as of TLS 1.2 (2008) and used by default by most clients.
- Message authentication (via GMAC/GHASH) is done on the ciphertext. (This is desirable most of the time.) Note that in most implementations, the auth check and decryption happen in parallel for performance reasons.
- Performance costs 1 x AES operation and 1 x GHASH per block (GHASH generally faster than AES, so GCM is faster)
- Encrypt/decrypt of multiple blocks can be parallelized nicely
GCM should be considered superior to CCM for most applications that require authenticated encryption. Because of the authentication that happens, GCM is not susceptible to the bit flipping and other attacks that can be mounted against counter mode or other stream modes.
There are some nuances that should be noted before using GCM that involve maximum size of the encrypted message and the MAC size. These are called out in the standard (NIST SP800-38D).
Using ZFS 2.0.3 with Arch Linux 5.10.19-1-lts on an old Intel Core2Duo E6600 (bought in 2006) without AES-NI I see a major performance benefit by using CCM. I did benchmarks with reading and writing blocks of data to a ZFS mirror using two RAM disks. The results show similar performance for aes-128-gcm and aes-256-gcm. aes-256-ccm shows up to the doubled throughput compared to aes-128-gcm and aes-128-ccm shows up to tripled throughput compared to aes-128-gcm.
Data was written to RAM and it was read from RAM to /dev/null. For the reading data rate I used the data rate of the first read. Follow up reads are much faster due to caching. dd was used for file access without input caching and with direct output access.
Setup 1: Data source is /dev/zero
Setup 2: Data source is GPG encrypted video data in RAM
From a pure performance point of view it may be good to run some benchmarks on your particular system before making a decision.
|Data||Access||ZFS mode||Rate (setup1) / MiB/s||Rate (setup2) / MiB/s|
|880 x 1 MiB||Write||noencr, nodedup, nocomp||170||171|
|880 x 1 MiB||Write||encrypt=aes-256-gcm, nodedup, nocomp||14.1||14.3|
|880 x 1 MiB||Write||encrypt=aes-128-gcm, nodedup, nocomp||14.6||15|
|880 x 1 MiB||Write||encrypt=aes-256-ccm, nodedup, nocomp||33.4||35.7|
|880 x 1 MiB||Write||encrypt=aes-128-ccm, nodedup, nocomp||38.3||41.9|
|880 x 1 MiB||Read||noencr, nodedup, nocomp||979||965|
|880 x 1 MiB||Read||encrypt=aes-256-gcm, nodedup, nocomp||15.2||20.2|
|880 x 1 MiB||Read||encrypt=aes-128-gcm, nodedup, nocomp||15.6||19.1|
|880 x 1 MiB||Read||encrypt=aes-256-ccm, nodedup, nocomp||38.6||40.3|
|880 x 1 MiB||Read||encrypt=aes-128-ccm, nodedup, nocomp||46.2||67.1|
|1 x 880 MiB||Write||noencr, nodedup, nocomp||156||202|
|1 x 880 MiB||Write||encrypt=aes-256-gcm, nodedup, nocomp||14||14.1|
|1 x 880 MiB||Write||encrypt=aes-128-gcm, nodedup, nocomp||14.3||13.7|
|1 x 880 MiB||Write||encrypt=aes-256-ccm, nodedup, nocomp||32.5||34.5|
|1 x 880 MiB||Write||encrypt=aes-128-ccm, nodedup, nocomp||38.3||40.1|
|1 x 880 MiB||Read||noencr, nodedup, nocomp||600||396|
|1 x 880 MiB||Read||encrypt=aes-256-gcm, nodedup, nocomp||14.9||17.5|
|1 x 880 MiB||Read||encrypt=aes-128-gcm, nodedup, nocomp||15.6||20.2|
|1 x 880 MiB||Read||encrypt=aes-256-ccm, nodedup, nocomp||29.8||48.4|
|1 x 880 MiB||Read||encrypt=aes-128-ccm, nodedup, nocomp||45||55.6|