File Hash of Encrypted File

Simple question. I have a source file and I have encrypted it with a certain public key. If I take this exact same source file, encrypt it with the exact same key a few hours later would the file hash of the two encrypted files be the same?

-

The short answer is NO: you will not obtain identical ciphertexts from the same file encrypted using two different runs of a decent public key file-encryption software; their hash will not coincide.

If you use a deterministic public key encryption scheme like textbook RSA, i.e., without randomized padding, then yes.

If you use a secure public key encryption scheme such as RSA OAEP (providing at least IND-CPA security), i.e., encrypting the same message twice with the same public key produces different ciphertexts with overwhelming probability, then no.

However, in the latter case your randomness used for encryption may be computed pseudorandomly from some known value (such as the message itself), as for instance used in message-locked encryption, then the answer turns to yes again.

Hybrid encryption:

If you encrypt larger files (larger then the supported message space of the public key encryption scheme) with a public key encryption scheme you should use hybrid encryption. This is also what will be used by a file-encryption software.

Let us denote the public key as $pk$ and the document as $m$.

• Then encrypting the first time results in $(c_1,c_2)=(E_{pk}(k),E_k(m))$ for a random symmetric key $k$.
• The second encryption will result in $(c_1',c_2')=(E_{pk}(k'),E_{k'}(m))$.

If you set $k=k'$, i.e., use the same symmetric key, then $c_2=c_2'$ (depending on the mode of operation used for the symmetric cipher you may also require to use the same IV/nonce). Otherwise, if $k\neq k'$ then $c_2\neq c_2'$ with overwhelming probability.

If you use a deterministic public key encryption scheme, then $c_1=c_1'$. If you use a probabilstic (secure) public key encryption scheme, then $c_1\neq c_1'$ with overwhelming probability (as said above, one however can turn probabilistic public key encryption deterministic public key encryption).

Hashing the ciphertext

Now, the first part of the answer only refered to deterministic/probabilistc encryption. It additionally depends on what you see as the hash of the ciphertext, or if you hash the file as it is, and what you want to guarantee in your application and what you want to store.

• If you only store the hash of the second component $c_2$ when using hybrid encryption, then the hashes will be identical if you use the same symmetric key (as well as IV/nonce), i.e., $k=k'$, and the document does not change.
• But why then use public key encryption? You could encrypt $k$ using $pk$ (store it with the encrypted file) and then decrypt and encrypt the file again and make the comparison.

• Alternatively, you could in addition to the file store the encrypted hash of the file.

• You could also simply store a hash of the unencrypted file.

• You could also simply ignore hashing and compare the encrypted files.

-
@TimKennedy I made an edit to include hybrid encryption. –  DrLecter Dec 12 '13 at 15:03
@figlesquidge feel free to edit if you want to add something ;) –  DrLecter Dec 12 '13 at 15:13
Will do in future (I'm always wary of modifying other peoples answers in any non-trivial way) –  figlesquidge Dec 12 '13 at 15:21
I suggest adding at the beginning something simple like: "The short answer is NO: you will not obtain identical ciphertext files from the same plaintext files encrypted on two different runs of a decent public-key file-encrypting program; their hash will not coincide.", with later justification that decent public-key file-encrypting programs use hybrid encryption. –  fgrieu Dec 12 '13 at 16:08
Thanks. This has been very helpful and informative. –  Tim Kennedy Dec 12 '13 at 18:21