Blake2 vs md5 for checksum / file integrity

Question

I'm working on something that requires computing a checksum / file integrity. I've heard MD5 was no longer recommended. Should I use Blake2 instead? Is there a specific variant of Blake2 I should use for checksums? Blake2s? Blake2sp?

Answer 1

Yes, you should choose BLAKE2 instead of MD5. Any of the BLAKE2 variants—BLAKE2s, BLAKE2b, BLAKE2sp, BLAKE2bp—is an improvement over MD5, even if you just use them for 128-bit digest sizes like MD5, although it is generally better to use at least 256-bit digests. BLAKE2b is a good default; BLAKE2s is slower on 64-bit systems but friendlier to 32-bit systems; BLAKE2sp and BLAKE2bp are optimized for systems with large vector units or multiple cores to parallelize hashing long messages.

So go and rip out MD5, and put in BLAKE2 instead, and you'll almost certainly be be better off.

---

You've ripped out MD5? Good! But we should also talk about what you're trying to do with it in the first place, because while BLAKE2 is not worse than MD5, there are things neither MD5 nor BLAKE2 nor anything with the same shape—a fixed hash function—can do no matter how good they are at what they do do.

So, when you say ‘checksum’ and ‘file integrity’, what are you trying to do? Here are some possibilities:

• You're trying to detect random errors with a very simple distribution, like independent errors, or burst errors.

  In this case, while BLAKE2 is not worse than MD5, you might be better off with something designed to be a checksum like a CRC rather than something designed to be a random function like BLAKE2 and MD5:

  • A CRC polynomial with a factor of $x + 1$ is guaranteed to detect all errors of odd parity.
  • An $n$-bit CRC with a nonzero $x$ term is guaranteed to detect $n$-bit burst errors.
  • Various other errors can be guaranteed detected by a judicious choice of generating polynomial; see the standard Koopman–Chakravarty reference (preprint, paywall-free) for more details.

  This is relevant when there is no adversary involved.

• You're having a conversation with someone with whom you share a secret $k$, and you want to prevent anyone else from forging messages.

  In this case, you actually need a message authentication code, or MAC, which has the property that anyone who doesn't know the key can't find the MAC for any message except ones they've already seen by asking you politely to give it to them.

  BLAKE2 has a built-in MAC: you can specify a key, and it magically turns into a MAC. There are other alternatives, like HMAC-SHA256, or even HMAC-MD5 (which is to the best of our knowledge still secure up to well below $2^{64}$ messages, but you should consider switching away just so that you don't have MD5 in your code at all).

• You're publishing statements that you want anyone in the world to be able to verify, but nobody to be able to forge.

  In this case, you actually need signatures, which are like MACs except there are separate keys for signing (private key) and verifying (public key). In principle BLAKE2 could be used in a signature scheme, but in practice I haven't seen any, and any good signature scheme for practical use will come with predetermined choices for any hash functions, like Ed25519 uses SHA-512 and Ed448 uses SHAKE256.

  Of course, anyone who wants to verify a message needs to know the public verification key a priori: if they got it on the same channel as the signed message, the adversary could have replaced the whole thing—but at least the recipient will have the public key for next time, so the adversary would have to replace the whole thing on every day the recipient tries to fetch the public key and/or signed message.

• You're publishing a manifest of files in a distribution, like a software package repository, and the manifest is published on a different channel—known to be unforgeable, e.g. because you got it over SSH or HTTPS from a good server—from the software packages themselves—e.g., because you download them over HTTP from a sketchy CDN.

  In this case, BLAKE2 is a good choice! If you used MD5, you might be in trouble. But you should also be sure that there actually is a qualitative difference in security between the channel on which you transmit the manifests and the channel on which you transmit the files; if an adversary can modify both at the same time, the hashes accomplish nothing.

• You're using a hash as a key in a content-addressed storage scheme, like Tahoe-LAFS.

  In this case, BLAKE2 is a good choice! If you used MD5, you would definitely be in trouble.

Answer 2

If in doubt, use BLAKE2, not MD5.

If security is necessary, then you should avoid MD5. It has a weakness where someone can create two files which differ, but which have the same MD5 digest. This is called a collision attack. If you simply want to check for duplicate files or files that are accidentally corrupted, then MD5 is still acceptable.

BLAKE2 is a newer hash with much better cryptographic security. BLAKE2s has a 256-bit output, meaning it provides 128-bit collision security, which is plenty. BLAKE2sp is simply a version which supports being run in parallel. Using BLAKE2sp would allow you to hash a single file using multiple hardware threads (processor cores) at once, which can give you a significant performance boost.

Answer 3

BLAKE2b is an excellent option for a fast, modern hash function. BLAKE2b is optimized for 64-bit platforms, while BLAKE2s is optimized for 32-bits and lower. The p versions are optimized for parallel cpus.

Most applications using BLAKE use the BLAKE2b variant, but either will be secure, far more so than MD5.