How to make a function which produces a hash of variable length?

Question

I have been learning about OTP lately and I wondered, if I was to use it for encryption of a file (like a photo or even a video), I would need really looong key.

Example program is simple: input is password and file name, output is encrypted file.

One of my ideas was to create a hash of the password, but just 512 bits (I am using sha512) are still shorter than average file. My final solution was to take hash of the password, divide it into two halves (first 256 and last 256 bits) and sha512 each one of them (this doubles size of the key). Then, for each of the resulting hashes, I repeat the process untill I reach the length of the file.

When I get to the length of the file, I just use the resulting hashes concatenated together to form a key with which I then encrypt the file.

Implementation in Python:

def get_key(passwd, filename):
    size = os.path.getsize(filename)
    pwd = passwd + filename
    
    initial_key = sha512(pwd.encode())
    subkeys = [initial_key[32:], initial_key[:32]]
    
    while len(subkeys) * 32 < size:
        new_subkeys = []
        for k in subkeys:
            k_hash = sha512(k)
            new_subkeys.extend([k_hash[32:], k_hash[:32]])
        subkeys = new_subkeys
    
    key = b''.join(subkeys)[:size]
    
    return key

This method is, however, relatively slow. Therefore, my question is, is there a better way of producing "hash" of length equal to the length of a file?

Is my method of dividing and concatenating hashes even secure?

Note about the One time password: I would probably concatenate the password with absolute path of the file before hashing, so that there are no reused keys.

Answer 1

This is the keystream generation step of a stream cipher, using a password as key. As far as I understand, it hashes the password using SHA-512, then expands the 64-byte result by rehashing every 32-byte segment (doubling the keystream size) as many times as necessary to reach the plaintext size; then the left of the resulting keystream is kept.

This is not a One Time Pad, for two independent reasons:

• The keystream is generated algorithmically from the (typically shorter) password rather from a true random source.
• Nothing stated insures One Time use of the keystream (see 1 below).

There are two serious security issues:

1. There is no Initialization Vector. Thus the XOR of two (equal size) ciphertexts made using the same password is the XOR of the corresponding plaintexts, allowing known-plaintext attack, and even ciphertext-only attack when the plaintext is redundant. The particular keystream expansion strategy makes it even worse: knowing a 64-byte plaintext and corresponding ciphertext, we can trivially decipher ciphertext of any size produced with the same password.
2. There is no key stretching. Therefore, it's possible to quickly test a candidate password, opening to brute force password cracking. The ability of humans to remember passwords is limited (to 44 bits by some estimate), and not growing. Ability of computing stuff to perform brute force password search grows about exponentially (doubles/increases by 1 bit every 1 to 5 years depending on who you ask), and therefore the security of password-based cryptography without stretching is going from bad to worse.

Performance of the question's code is poor for a variety of off-topic comp-sci reasons, including an ugly transformation to hex and back to bytes in the end.

If we stick to a single standard algorithm available in Python without add-on, we can use scrypt (available as hashlib.scrypt), which includes memory-hard key stretching, and has variable output size (with that later feature using PBKDF2-HMAC-SHA256 reduced to 1 round).

from hashlib import scrypt

# generate a keystream of size bytes from password and a (unique) IV.
def get_keysream(passwd, IV, size):
    return hashlib.scrypt(passwd, salt=IV, n=32768, r=8, p=4, dklen=size)

The above uses a little over 128*32768*8*4+size bytes of memory (128kiB + output size), and up to 4 threads.

For large files, we'd want to split the password-to-key transformation (using scrypt or Argon2) from the keystream generation (e.g. with ChaCha or Blake2b), done in chunks of bounded size and with ability to generate arbitrary chunks, to allows direct access to encrypted files (necessary e.g. for fast navigation in encrypted video).

Answer 2

As I understand the question, the intent is to use a "password" as a key, and expand this key by generating a series of hash values and concatenating them to be used as keying material for OTP encryption.

The scheme used in that sense would be insecure if used more than once with the same password.

If you use the same password to encrypt two messages, one can calculate the XOR of the two encrypted messages to recover the XOR of the original messages (your key cancels out). If there is any structure or redundancy in the original messages, they can likely be recovered (at least partially).

A more typical construction is to use a secret key the same length as the hash, say $k$, then pick a random number, a nonce, $n$ (for simplicity, I will set $n$ to be the same size as the hash).

You generate your OTP as the concatenation of a list of $x_i$, for $i=0\ldots l$, where $$ x_i = \mathtt{Hash}(k \vert n+i)$$

The "$\vert$" denotes concatenation.

This construction is not proven secure (to my knowledge) but is generally accepted as secure when using a "good" cryptographic secure hash function such as sha256 or sha512.

You will need to send your nonce $n$ in the clear along with any data processed with this OTP. That's ok; just don't send your key! Pick a new nonce every single time you use your key.

Note that OTP encryption preserves confidentiality very well, but is susceptible to all sorts of manipulations; it is not resistant to tampering attacks.

Answer 3

I too play with OTPs. I use batteries, diodes, cameras and a soldering iron. I also leverage the Left Over Hash lemma producing $\epsilon < 2^{-10,000}$. And it is slow. Yet I don’t have a new_subkeys.append() thingie.

The “Strategic Defence of Taiwan” with ‘nuclear’ somewhere towards the ending may be considered as an important document. It might even be hush-hush and only transmitted via OTP. And yes, you will need a really looong key, but much shorter than that for an 8K UHD video of frolicking pussies.

Please research what a one time pad is /for as it's commonly misconstrued.