AES encrypting multiple files

Question

So if I want to encrypt all the files in a folder with AES (same password) I take each file and generate the key schedule using PBKDF2. As the PBKDF2 algorithm takes in a salt then this salt should change (and thus the keyschedule) for each new file being encrypted. Now if I want to decrypt then I load the plaintext salt and generate the different key schedule for each ciphertext file.

Is this correct? If so the problem I'm seeing is that the recommended number of iterations to make the encryption more secure (0.5 seconds e.g.) will considerably slow down the encryption process as each file requires a lengthy key derivation procedure. Is this just a necessary part of a secure encryption process?

Thanks!

Answer 1

You are correct that using a different salt for each file will slow down encryption and decryption, in proportion to the number of files. But it is not useful to do so. An adversary is not helped if she knows that the salt is common to several files hashed with the same password (under the assumption that she can recognize a correct password with fair probability from one ciphertext, and cost dominated by PBKDF2). Both with common and different salt per file, the hard work for the attacker is finding the password; the cost of recomputing PBKDF2 for other files when salt is different will be comparably negligible.

Thus an option if to have the enciphering program generate the salt at random common to every files, and appropriate IV for each file (e.g. 48-bit file number, and 80-bit zeros; this is enough for $2^{48}$ files of $2^{84}$ bytes each, encrypted using AES-CTR); compute only one key using PBKDF2; and store salt and IV as header of each enciphered file. The deciphering program can cache the key in RAM for reuse when deciphering files with identical salt (it should not assume all files have identical salt, in case files separately enciphered are merged into the same directory).

When using other block-cipher operating modes such as CBC, CFB or OFB for enciphering multiple huge files, you may approach the limit of reasonable usage of one key. For this reason, or just for added peace of mind, you might want to have one key per file. This can be done while still amortizing the derivation of key from password over multiple files, by generating a common "slow" master key from password and master salt (e.g. random) for each encryption session, and a "fast" derived key from master key and secondary salt (e.g. file index) for each file, with both salts in the encrypted file's header. The "fast" derivation could be with PBKDF2 and a minimal iteration count c=1, or just one HMAC.

Also: you might want to consider scrypt instead of PBKDF2 (at least for the master key), for it provides better security for a given time overhead, as summarized in this table from the scrypt paper (2009):

Update (2020): state of the art has evolved. Bitcoin mining has shown that dedicated ASICs performing SHA-256 at high rate (220 TH/s) and efficiency (15 pJ/H) can become commercially available for few thousand US dollars. This makes it untenable to rely on PBKDF2-HMAC run on standard hardware for stretching of user-chosen passwords in high-security applications. Scrypt still stands strong, however the new best practice for password hashing is the winner of the password hashing competition, Argon2.

Answer 2

There really isn't any measurable security in that. It's also inconsistent. If you had thirty text 1KB files, you'd be generating thirty new schedules, but if it were one 20MB video, you'd be using one schedule for all of the data.

Really it depends on your mode of encryption. For example, if you're using CFB the iv would change depending on the previous block anyway, so there's no reason to keep switching schedules.