Prior source material studied
Question & Details
Assumed threat model
Prior source material studied
0.1. RFC 2898
0.2. HAC (Handbook of Applied Cryptography)
0.3. Crypto 101
0.4. SE answers (ex: If attacker knows salt and hash, how is salt effective?) and (Use of salt to hash a password)
0.5 2018 DBIR
How is a KDF using a known salt and a low-entropy password more secure than the so-called "completely broken and not at all secure" primitive hash with salt?
Based on the available subject material, the KDF salt is not required to be kept a secret.
Does the recommendation imply a starting assumption in the threat model that the user-provided secret-key/password will always have sufficiently high entropy, or won't be an instance of weak one? Or does the recommendation only apply to specific instances where the same or different data is encrypted regularly with a different salt/key each time?
The author of Crypto101 writes this about generic Crypto Hash Primitives:
"While this [rainbow tables based on a known single salt] would have been prevented by using a different salt for each user, systems that use a cryptographic hash with a per-user salt are still considered fundamentally broken today; they are just harder to crack, but not at all secure."
If the threat model always assumes (as it should) that the user-provided secret key could be weak, then how does using KDF with a known salt become more secure than the aforementioned broken pattern of using a Hash Primitive with unknown (high-entropy) salt? (see Clarifications for answer to an obvious counter here)
(Aspects of) Assumed threat model for the question:
2.1 The user creates a low entropy password (TW37VE), which needs to be stored, and used for login.
2.2 The threat actor will use the same kdf (with correctly-guessed iteration) to compute the key from this common low-entropy password.
2.3 Salt needed to add entropy to the user-provided password (edit: a typo originally stated 'secret key' instead of password here) -- which is assumed to be from low-entropy space -- will be assumed here to be known to the threat actor (based on the recommendations in the documents).
2.4 The slow-computation/high-memory cost of KDF is bearable, if only for access to even a single user data: for instance, it could be a case of an insider-threat actor willing to access data for a specific user.
2.5 Internal/Insider threat actors: Based on 2018 DBIR, 28% of all the reported data breaches involved an insider/internal actor. The threat model thus accounts for an internal actor as a potential threat even in the absence of an external one. Thus, the need to secure the salt as well. (See clarifications for answer to the obvious counter)
Say for password TW37VE, PBKDF2_SHA256 generates a key with the random salt 0CE@N's, and 100000 iterations, such that the derived key is:
kdf(TW37VE, 0CE@N's, 100000) : pbkdf2-sha256$100000$OCE@N's$!@#%^&
Threat KDF: kdf(harvested-weak-password = TW37VE, not-secret-salt = 0CE@N's, guessed-iterations = 100000):
If a legitimate user could validate itself using the known low-entropy password and the retrieved salt, what would be the deterrant in case of an attacker armed with (a) the harvested password (or it's precomputed dictionary/rainbow hash), and (b) the not-kept-a-secret salt?
Even if it is trivial for the threat actor to scan-and-match the password against the, say 64-bit, salt key space, the resulting potential delay wouldn't hurt.
Assumptions made in the question:
4.1. The non-random part of the derived key is irrelevant to the security model strength.
4.2. The attacker could just as well use pbkdf2-sha256 with the known salt, the harvested secret key, and the correctly-guessed iterations.
5.1 The drawbacks of using certain primitive-based approaches are understood; and, so are and the advantages of random-salt KDF's in differentiating similar contents.
5.2 The question focuses on why a primitive-based pattern under the assumed threat model is said to be broken, whereas the KDF pattern which simply yields to similar threat models under similar conditions is considered as an improvement over the former.
5.3 The DBIR reported "28% internal actors" figure includes different types of actors with different actions, which -- although likely to be small -- in this case is assumed to be a developer from the application-server team gone rogue (or maybe it was always bad?).
5.4 The home-grown roll: there is a way to keep the salt encrypted without the threat actor having the access to the salt-decryption key unless the entire database server is compromised.
To reiterate, how does NOT keeping the kdf salt a secret help, when exposing it could have the potential to be harmful, and security-equivalent of an unsecure primitive method?