Considerations on using a single long-term key for file encryption

Question

I'm thinking about designing my own container file format that allows me to store a file in it along with, say, a thumbnail or other metadata. Some of these files are confidential and therefore must be encrypted. Basically, my goal is to achieve confidentiality, authenticity, and integrity for my file containers. But, with possibly millions of containers and only a single long-term secret (i.e. master key file), how can I do that properly?

I suppose what I'm looking for is AEAD (AD = the file header) for which I can, e.g., use AES-256-CBC + HMAC-SHA256 or AES-256-GCM.

The issue I'm held up with is the IV/nonce. Since there is only a single long-term secret in my design, I understand that I must ensure the uniqueness of the IV/nonce for each container with either one of the methods mentioned above.

• Is it sufficient to use a simple sequence number for each container and place it in the (authenticated) file header?
• Should I split the IV/nonce in a fixed and a sequential part (mentioned in RFC5116)?
• Different approach?

I was also thinking about using an HKDF to derive key material for each container from the master secret and a GUID associated with each file container. I would then store the GUID in the plain file header. But then again I could simply use the GUID as IV/nonce. Is there a benefit in creating individual key material for each container?

---

Update

Individual keys for each container seem like the way to go. However, simply using a master secret to encrypt the container key does not solve the IV problem. Please take a look at the following proposal:

File header

$H: Plaintext\ file\ header$
$G: GUID\ associated\ with\ the\ payload\ (included\ in\ H)$

Payload

$P: Plaintext$
$C: Ciphertext$
$IV_p: Payload\ IV$
$K_p: Payload\ key$

$C = AES(IV_p, K_p, P)$

User keys and MACs

$S_i: A\ secret\ stored\ in\ a\ file$
$IV_i: IV\ for\ slot\ i$
$K_i: Key\ for\ slot\ i$
$K_{M_i}: HMAC\ secret\ for\ slot\ i$
$M_i: HMAC\ for\ slot\ i$
$K_{P_i}: Payload\ key\ material\ encrypted\ with\ K_i$

$(IV_i, K_i, K_{M_i}) = HKDF(G, S_i)$
$K_{P_i} = AES(IV_i, K_i, IV_p\ ||\ K_p)$
$M_i = HMAC(K_{M_i}, H\ ||\ C\ ||\ K_{P_i})$

Container file format

$H\ ||\ C\ ||\ (K_{P_i}, M_i)$

With this design, I

• use a HKDF to derive as much key material from the keyfile as I need (solving the IV problem)
• can later on add or remove as many user keys as I want
• can detect tampering via the HMAC associated with each user key
• do not need to store a hash of the container key (like LUKS does) because the HMAC guarantees the integrity of the decrypted key

Would you consider this a sound design?

Answer 1

Is it sufficient to use a simple sequence number for each container and place it in the (authenticated) file header?

Yes, but making sure it is unique may be trickier than you think. Storing a counter is both error prone and it may hamper performance as well. What if you lose the .lock file (or similar) containing the number?

Should I split the IV/nonce in a fixed and a sequential part (mentioned in RFC5116)?

If you want to, as long is it stays unique. This part of the standard seems particularly important if you've got multiple devices that each have their unique fixed ID. In that case the fixed ID can prevent the devices with coming up with the same nonce.

Personally I don't think you should use the same key in the first place. I don't consider RFC 5116 a strong standard and I would certainly not use it to draw cryptographic ideas from. Especially the part where the authentication tag is part of the ciphertext feels completely backwards to me (as it removes the online properties of AES-CTR that is used for most authenticated modes of encryption).

Different approach?

Usually a symmetric key is generated for each file, which can then be wrapped by the master key. The advantage is that you can decrypt the data key and add other keys that may retrieve the data key. You may also be able to roll over to a new key that way. In other words, it simplifies key management. That kind of key management is harder to accomplish when using a KDF.

But then again I could simply use the GUID as IV/nonce. Is there a benefit in creating individual key material for each container?

Yes, you'd be less vulnerable against catastrophic failure if the nonce ever repeats.

---

Be warned that you may want to protect against files being swapped as well.