For data integrity, specifically for small blocks of data, rather than using a SHA-256 hash, or simply truncating a SHA-512 hash, I am playing with the following idea:

function saferHash(data) {
    let hash1Entropy = 42;  // sha512:42
    let hash2Entropy = 4; // 4B extra possible protected outcomes in case of highly unlikely collision in hash 1
    const hash1  = sha512(data).slice(0,hash1Entropy); // for 42 byte entropy, truncated to save space + prevent length extension vulnerabilities 
    const hash2 = blake3(data).slice(0,hash2Entropy); // cheapest extra forgery prevention in case of sha512:42 hash collision, truncated to save space + prevent length extension vulnerabilities 
    return new Uint8Array([...hash1,...hash2]); // 46 byte total hash  

Is this totally overengineering the whole thing or is it actually not too bad of an idea to increase the entropy to +32 bytes and use an additional cheap hash function (blake3) to increase the difficulty of creating a collision even more?

  • $\begingroup$ Well I'm not sure if taking some of each hash instead of just taking a full hash reduces collision rate. After all, when you truncate each, you increase the collisions accordingly. I bet eventually the collisions will be the same, but less chance to appear in rainbow-tables as it's unique. $\endgroup$
    – Bharel
    Jul 5, 2022 at 19:30
  • $\begingroup$ Well, when a collision is found in hash1 (SHA-512) it does not (in 4B cases) cause a collision in the full hash (hash1 + hash2), right? $\endgroup$ Jul 5, 2022 at 21:05
  • $\begingroup$ Maybe on a side note, it's worth to mention the potential 'risk' of a researcher that one day finds an effective algorithm to create collisions in one of the used hashing algorithms, for example SHA-512. $\endgroup$ Jul 5, 2022 at 21:32
  • 1
    $\begingroup$ Given that CPU power and storage are virtually infinite yet cost zero, why is this a concern? $\endgroup$
    – Paul Uszak
    Jul 6, 2022 at 19:47
  • $\begingroup$ Well, if I increase the CPU+storage my monthly VPS bill definitely goes up. $\endgroup$ Jul 6, 2022 at 19:51

4 Answers 4

  1. Yes, 32 bytes/256 bits is considered enough (seriously read this blog post). That length provides $2^{128}$ collision resistance and $2^{256}$ pre-image and second pre-image resistance for SHA256 and so on.
  2. 48 bytes/384 bits or 64 bytes/512 bits provides greater security and would be more sensible to use under certain circumstances but unnecessary in others. For example, if you were to prehash a message for Ed25519, you would want to use a 512-bit hash to maximise collision resistance.
  3. This is overengineering. What you're doing is weaker than using regular SHA512 because the collision resistance depends on the length of the output. 4 bytes of output is excessively small and should not be used. The default output length for BLAKE3 is 32 bytes/256 bits, and you should generally not go lower than that for any unkeyed hash function. 42 bytes is also an odd choice. There's no need to use multiple hash functions or reinvent the wheel. If a single hash is good enough for well-established real-world protocols, it's good enough for you.
  4. To prevent length extension attacks, either use SHA512/256 or ideally a more modern hash function, such as BLAKE2, SHA3, or BLAKE3 (in that order in my opinion). You don't need to truncate BLAKE3 to avoid length extension attacks.
  • $\begingroup$ Why would SHA512/256 be better for preventing length extension attacks than SHA512/42 bytes? $\endgroup$ Jul 5, 2022 at 21:09
  • $\begingroup$ Also agree on 3, but it's seems better than SHA-256, and better than full SHA-512 in regards to storage space. $\endgroup$ Jul 5, 2022 at 21:11
  • $\begingroup$ @swordsecurity SHA512/256 is an actual algorithm, whereas SHA512/42 bytes is not. 42 bytes is a very odd output size that I've never seen used. 256, 384, and 512 are the standard output lengths. $\endgroup$ Jul 5, 2022 at 21:30
  • $\begingroup$ It's indeed far from standard. The 42 byte size is mainly chosen to gain space in comparison to SHA512 or SHA512/384 and to gain entropy in comparison to SHA-256. A sort of "best of both worlds" from my perspective. $\endgroup$ Jul 5, 2022 at 21:36
  • $\begingroup$ @swordsecurity Also, I would not call it 'better than SHA256'. The BLAKE3 part is pointless. If you want to save storage space, what's wrong with 48 bytes (384 bits aka SHA384)? That's stronger than 42 bytes, barely any difference in storage, an ordinary output length, and SHA384 is resistant to length extension attacks. $\endgroup$ Jul 5, 2022 at 21:36

By truncating hashes, you actually increase the collision rate of each.

Eventually for a certain amount of bytes, and considering the algorithms are static and are relatively secure (by having a close-to-uniform output), the collision rate will stay more or less the same.

The only thing you prevent is probably rainbow-tables, should ones exist for a certain hash, most won't exist for your particular combination.

All-in-all I believe this is over-engineering and it's safer, more convenient and definitely faster to fully use sha-512 than truncate each and reinvent the wheel.

In simple terms you're truncating a slower and "better" algorithm, reducing the "good" entropy you receive from it, and replace it with a "cheaper" algorithm.

  • $\begingroup$ Definitely agree on the reducing the "good" entropy part, yet the truncation is done to both safe space and prevent length extension attacks. Also it seems when a collision is found in hash1 (SHA-512) it does not (in 4B cases) cause a collision in the full hash (hash1 + hash2), right? $\endgroup$ Jul 5, 2022 at 21:06
  • $\begingroup$ Also agree that SHA-512 would be "better", yet full SHA-512 is 64 bytes, which is takes up more storage space than the proposes 46 bytes. $\endgroup$ Jul 5, 2022 at 21:13
  • $\begingroup$ @swordsecurity Why not truncate the SHA-512 to 46 bytes then. Truncating 18 bytes would already prevent a length extension attack. If you use 2 truncated hashes, you will basically cause a full hash collision at the exact same rate. After all, those bytes are meaningful. $\endgroup$
    – Bharel
    Jul 5, 2022 at 21:38
  • $\begingroup$ Optimally I'd love to just use SHA-512, yet I don't know if the "exact same rate" of collisions is true. After all they use an entirely different method to arrive at their outputs? $\endgroup$ Jul 5, 2022 at 21:40
  • $\begingroup$ @swordsecurity it doesn't matter. Let's say theoretically the output is completely random. 10 bytes of random data is equivalent to 6 + 4 bytes of random data. The collision rate is exactly the same. $\endgroup$
    – Bharel
    Jul 5, 2022 at 21:56

The size of a hash depends on your expected cryptographic needs. We expect 128-bit security to be sufficient for the indefinite future. As such, a 32-byte (256-bit) hash should be fine for classical cryptography. However, if you need collision resistance that will survive post-quantum cryptography, then a longer hash (384 bits or more) is suggested.

My recommendation is to avoid mixing and matching hashes because that doesn't always add additional security like you think it does. It is better to use more output from a single, strong hash than mixing and matching hashes. This also makes it easier to follow cryptanalytical results, since the construction is simple and standard. Similarly, if your code needs to undergo an audit for any reason, it's much easier to simply say, "I'm using this standard hash algorithm," then to explain to the auditor why you chose a custom construction.

In general, any of the SHA-2 or SHA-3 algorithms of sufficient size or BLAKE2 are good choices. BLAKE3 may also be an option, but it has seen less cryptanalysis and is differentiable from a random oracle. If you want a very fast option, BLAKE2b is faster than MD5 and SHA-1 and is cryptographically secure, and it can be configured to provide any output size up to 512 bits (although some implementations provide only that size). You can truncate an algorithm, but I would recommend simply choosing one which meets an appropriate output size.

If you are really truly worried about cryptographic developments, you can use a format like multihash to store your data and then, if necessary in the future, simply transition over to a new function.

  • $\begingroup$ Thank you so much for suggesting multihash!! $\endgroup$ Jul 6, 2022 at 12:15

(A truncated version of) one secure hash (like SHA-512) seems to be a viable approach, possibly 32 bytes long, possibly a few bytes longer if you want to increase collision resistance.

When concatenation multiple hashes the "weakest" hash function may actually be used to determine and crack the message, since you could simply brute-force hashes faster (decrease preimage resistance).

Especially in the case of BLAKE3 (which is benchmarked to be much faster than SHA-512).

See also https://security.stackexchange.com/questions/83881/is-using-the-concatenation-of-multiple-hash-algorithms-more-secure


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.