Line by line encrypted logging stored with iv/salt/iterations. How safe is it?

Question

I'm building an encrypted logging application. Log entries are encrypted with AES-256-GCM, using password-derived keys with PBKDF2. The application must support log rotation, and I would like to make each log entry "self sufficient". This would allow me to discard previous logs that have already been processed/are too old.

By self-sufficient, I mean that having the log entry and the password is sufficient to decrypt the payload. I do not wish to maintain a separate file/record with information about the key or its derivation.

That means I have to store a whole bunch of information with my each log entry:

• pbkdf2 salt
• pbkdf2 iterations
• iv
• gcm tag
• encrypted payload

The PBKDF2 salt and iterations are not likely to change often, if ever. The IV will be randomly generated for each log entry. The payload is user-provided content The tag is the verification tag for the GCM encryption.

Here's an example of such a log file:

timestamp  pbkdf2 salt              iter   iv               tag                      encrypted payload
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,IxsZwVEjYvO0vgjl,w4EdRjRJQNCK/sOYKYyXYA==,DCqFFCGQbpYhWEy3u3OlKMN2iMn9JF2vFZFY
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,n1PscNktuug1Os7b,e1oZAg31XCZOZUC4ZrKjzg==,cLhr1baGonaTt/k/PJZ2s5hboTBjNCmqFu281w==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,RqKtky2ICMORWqGZ,G5VHGDkhWuUch7ZN0AiRBg==,mjexQ9aUHCZGomrlrnQrk6K9ULBEYHPSsjeAxw==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,JY3h/TQSlk+yP/5W,bXL8xK5QVC9uPi2vJ+IDQw==,xu5e/UKPyeEnghFVJL+JLmV2yXYlN2cKBmGGZw==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,W/3uKytKkGmPezdP,HuGNKT6L+Bh/bEvwnlkI5A==,rwVB3ssIKNaz0ZldNRR608JKOIjlimnuu5ZAyQ==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,l7RSko8EvTydWWgt,Ahwjm63IR/8MK03gn//uQQ==,cVS+O0nP7BrhuFCheXn1EDSdGSXumzu8PL1cUQ==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,H+5ywMwpalZVPwLj,nzVSRSz08gJ+OxcktniAOQ==,4WbpNcfcIEcx0wDyzH7s40u+Utx+xzlIFfosuA==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,hF5i+nCT3lFFxHJh,w9MYSR4j3tlebur+ZkUyVg==,9ZIlK8ooUhV0AVAJso6DIstAZ3IdxXCbNhpG4A==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,RAqAaY4cXsAMXZ3m,JzS4uU4QuMeenWqOMnCoRA==,phjLHaBeETJ+H5mRq0i1EUIe/MIbNmxEnNhLcA==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,iGXfpm7FZ7Rmj0F1,TRi0/ElsdCbdUUGBrhxqhQ==,P2yMW+1pn9F7kTTU7bWOm19xrjWQU+0UuhJGKw==
1584815488,IqdD09R5kbtA4recw5LQeg==,100000,IXNxT6RwDu3BnoFB,7khAMRKSLcE6nAcxdUsQbQ==,FMv3gUuO5Q2HVwetr/uo73ph4UOuA+Z3IOiXEg==

1. Is it safe to store all of that information at the same place?
2. A log file will typically contain hundreds/thousands of such entries. Does the encryption get significantly weaker as entry count increases?
3. The timestamp has to be stored in cleartext, for practical reasons. However, I would like to validate that it hasn't been tampered with. What's the best approach for this?

Answer 1

1. Is it safe to store all of that information at the same place?

All the information (salt, pbkdf2, iterations, iv, GCM tag, encrypted payload) is listed can be considered as public information. We already consider that an attacker has the knowledge of all of these but the encryption key by the Kerckhoffs's principle.

2. A log file will typically contain hundreds/thousands of such entries. Does the encryption get significantly weaker as entry count increases?

The IV is generated randomly. If the random generator is weak then it might cause an IV reuse and this is a big security issue under the same key for AES-GCM due to the underlying CTR mode and that can reveal the plaintext by crib-dragging and even more it can cause forgeries.

The IV can be generated with a counter or LFSR as recommended by NIST 800-38d 8.2 IV Constructions so that it will never repeat. A better idea is half counter/LFSR half random.

$$\text{nonce} = \text{random} \mathbin\| \text{LFSR}$$ This can prevent a system failure issue that the last stage of the counter/LFSR is not stored then it can cause a nonce repetition. Although one may repeat the counter/LFSR stage, the new random will prevent the IV reuse issue.

AES-GCM has IND-CCA2, it is not getting weaker especially with a 256-bit key size. However, it might be a good idea to change the key over a period of time. This can be done using a master key and derive keys form using HKDF or using different salts for PBKDF2.

3. The timestamp has to be stored in cleartext, for practical reasons. However, I would like to validate that it hasn't been tampered with. What's the best approach for this?

The Authenticated Encryption (AE) is also known as Authenticated Encryption with Associated Data (AEAD). You can provide the timestamp as associated data and the AES-GCM will use it in the integrity part. That is designed for such purposes. This can be formalized as

$$(C,T) \leftarrow \operatorname{AES-GCM-}\mathcal{E}^{N,A}_K (M)$$ where

$\mathcal{E}$ stands for encryption, $K$ stands for key, $N$ stands for the nonce, $A$ stands for the associated data. C is the ciphertext and T is the authentication tag. Similarly the decryption;

$$(M | \perp) \leftarrow \operatorname{AES-GCM-}\mathcal{D}^{N,A}_K (C)$$ where $\mathcal{D}$ is the decryption and $\perp$ is the tag mismatch halt.

update per comment

What if the attacker had one line of that log file, and the corresponding cleartext payload. For example, knowing that line #1 of the example above says "Initial log entry". Does that compromise the key?

AES is primitive and it is supposed to be a pseudorandom permutation. It can achieve IND-CPA or IND-CCA or authenticated encryption using by using the appropriate mode of operation together with AES.

We assumed that one has solved the IV reuse, then in CTR encryption of AES-GCM, we have;

$$C_i = P_i \oplus \operatorname{AES}(k, \text{nonce}\mathbin\|\text{counter}_i) $$ and that means the attacker will know the plaintext and ciphertext blocks. This is actually the Known Plaintext Attack (KPA) and AES has believed to be secure about this. You can also read here the possible attacks on AES-128 and AES-256 far from it.