Length Extension attacks in SHA1 and MD5?

Question

From my understanding on the internet about length-extension attack, I have understood that hash(secret_key|known_data) can be exploited to produce hash(secret_key|known_data|appended_data) even without knowing the exact value of the secret_key by just knowing the length of the key. I tried implementing this using hashpumpy python module and this did not match.

import os
import sys
import hashpumpy

secret_key="secretkey"
known_data="data"
appended_data="append"

hash_1="9885f8af04289135df259e34bd22d17fe45ea81e" # hash of secret_key+known_data
hash_2="e9163e43652a921df34f76e635d32f6aad1286b8" # hash of secret_key+known_data+appended_data

new_hash,msg=hashpumpy.hashpump(hash_1,'data','append',len(secret_key))
print(new_hash,msg)  

Output:

475bdd96a25e7717842072c5be2944076ba31143 b'data\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00happend'

But the hash I want to calculate is of secretdataappend

Answer 1

You forget one little step of how Merkle–Damgård construction works; the padding, here SHA-1 padding:

append the bit $\texttt{1}$ to the message e.g. by adding $\texttt{0x80}$ if message length (ml) is a multiple of 8 bits.

append $0 \leq k < 512$ bits $\texttt{0}$, such that the resulting message length in bits is congruent to $$−64 \equiv 448 \pmod{512}$$

append ml, the original message length, as a 64-bit big-endian integer. Thus, the total length is a multiple of 512 bits.

Therefore the real input is

$$\text{SHA-1}(\text{secret_key}\mathbin\|\text{known_data}\mathbin\|pad1)$$

because the attackers only get the hashed value. And, you need

$$\text{SHA-1}(\text{secret_key}\mathbin\|\text{known_data}\mathbin\| \text{pad1}\mathbin\| \text{appended_data} \mathbin\| \text{pad2})$$

for extension attack. This works similarly for any hash function that uses the same padding mechanism.

The attackers can access the hash of a padded message and they can extend it in this way. This is due to the artifact of Merkle–Damgård construction. SHA-3 is safe from this attack by design and later the new SHA2 modes SHA512/256 and SHA512/384 and other truncated versions of SHA2 also have resistance to length extension attacks.

SHA-1 or MD5 is not considered secure. Use modern Hash functions like SHA3 or Blake2 with large output sizes like 256-bit to prevent generic collision attacks, classic or quantum.