I'm thinking about designing my own container file format that allows me to store a file in it along with, say, a thumbnail or other metadata. Some of these files are confidential and therefore must be encrypted. Basically, my goal is to achieve confidentiality, authenticity, and integrity for my file containers. But, with possibly millions of containers and only a single long-term secret (i.e. master key file), how can I do that properly?
I suppose what I'm looking for is AEAD (AD = the file header) for which I can, e.g., use AES-256-CBC + HMAC-SHA256 or AES-256-GCM.
The issue I'm held up with is the IV/nonce. Since there is only a single long-term secret in my design, I understand that I must ensure the uniqueness of the IV/nonce for each container with either one of the methods mentioned above.
- Is it sufficient to use a simple sequence number for each container and place it in the (authenticated) file header?
- Should I split the IV/nonce in a fixed and a sequential part (mentioned in RFC5116)?
- Different approach?
I was also thinking about using an HKDF to derive key material for each container from the master secret and a GUID associated with each file container. I would then store the GUID in the plain file header. But then again I could simply use the GUID as IV/nonce. Is there a benefit in creating individual key material for each container?
Update
Individual keys for each container seem like the way to go. However, simply using a master secret to encrypt the container key does not solve the IV problem. Please take a look at the following proposal:
File header
$H: Plaintext\ file\ header$
$G: GUID\ associated\ with\ the\ payload\ (included\ in\ H)$
Payload
$P: Plaintext$
$C: Ciphertext$
$IV_p: Payload\ IV$
$K_p: Payload\ key$
$C = AES(IV_p, K_p, P)$
User keys and MACs
$S_i: A\ secret\ stored\ in\ a\ file$
$IV_i: IV\ for\ slot\ i$
$K_i: Key\ for\ slot\ i$
$K_{M_i}: HMAC\ secret\ for\ slot\ i$
$M_i: HMAC\ for\ slot\ i$
$K_{P_i}: Payload\ key\ material\ encrypted\ with\ K_i$
$(IV_i, K_i, K_{M_i}) = HKDF(G, S_i)$
$K_{P_i} = AES(IV_i, K_i, IV_p\ ||\ K_p)$
$M_i = HMAC(K_{M_i}, H\ ||\ C\ ||\ K_{P_i})$
Container file format
$H\ ||\ C\ ||\ (K_{P_i}, M_i)$
With this design, I
- use a HKDF to derive as much key material from the keyfile as I need (solving the IV problem)
- can later on add or remove as many user keys as I want
- can detect tampering via the HMAC associated with each user key
- do not need to store a hash of the container key (like LUKS does) because the HMAC guarantees the integrity of the decrypted key
Would you consider this a sound design?