Extending the size of input for SHA-2 function

Question

In the question Does a hash function necessarily need to allow arbitrary length input? many answers talk about the theoretical limit of the SHA-2 functions being $2^{64} - 1$ bit due to padding. I was wondering, is there a secure way to extend this? From the description, it seems that the $2^{64} -1$ bits is not a security restriction, but more a padding issue.

Would it still be considered safe to have an SHA-2 function that chunks the input data into $2^{64} - 1$ bits of hash size and use the hash of the previous chunk as a salt for the next chunk? Are there other options to extend the size of the input of those hash functions to arbitrary length?

Answer 1

$2^{64-1}$ bits that make 2.30584301 exabytes ^*. If you are not restricted to SHA256, then use SHA512 that allows files to have size at most $2^{128}-1$, or use SHA3 that has no limit.

The NIST must use a limit due to the artifact of the MD construction. SHA256 is standardized in 2001 along with SHA512. They have internal block size 512 and 1024 respectively, in order to reduce the change of internal collisions, double the output size double the internal size double the file limit!

If you want to extend the limit of SHA256, then you are out of standards. SHA512 is already beyond all.

Would it still be considered safe to have a SHA2 function that chunks the input data into (2^64 - 1) - hash size bits and use the hash of the previous chunk as a salt for the next chunk?

Yes, you can extend with this $h_1 = \operatorname{SHA256}(m_1))$ and $h_2 = \operatorname{SHA256}(h_1||m_2)...$

But safe is a vague term, safe against what? Hash calculation is free, SHA256 has a length extension attack that you may need protection. In theory, I'll not talk about the numbers, you will get more chance of internal collisions, that can theoretically reduce the second pre-image resistance, or read here

Another alternative is the Merkle Tree.

Are there other options to extend the size of the input of those hash functions to arbitrary length?

Use SHA3-256, or better use SHAKE128-256 or and SHAKE128-512

---

^*What is Exabyte: An exabyte is $1000^6$ bytes.

This is a bit unrealistic to hash since;

• OpenSSL hash time is approx 450 Mb per second for SHA256. Hashing 1 Exabyte will take 70466 years on my setup.

• NSA's Utah data center is supposed to have 12 exabytes

• Online storage and service companies like Google, Amazon, Microsoft, and Facebook are estimated to have at least 1,200 petabytes of storage, 1.2 Exabytes.

Answer 2

The size is just restricted by a length encoding at the end of the last block that is hashed using SHA-256 - one of the two main hash functions that make up the SHA-2 family. If you extend that size then you'd have your secure hash function with extended input.

However, there is an easier way. The SHA-512 hash function - the other main hash function in the SHA-2 family - and all the derived hash functions - SHA-384, SHA-512/256 and SHA-512/224 - already use 128 bits instead of 64 bits to encode the length. That means that they can hash messages up to $2^{128} - 1$ bits.

SHA-512 also extends the state - the amount of hashing data transferred when going from one hash to another - so SHA-512 and its derivatives will be more secure as well. Then again, the cycle size of SHA-256 is probably large enough anyway.