-1
$\begingroup$

[Clarification edit further down.]

To distinguish already existing one-time pad files by their content instead of just their name ...

... is hashing meaningful? Or would the first X bytes (Length the hash would have.) have exactly the same representative value?

My guess is that the answer is

Reasoning edit:

The reason why I assume the answer is "yes" to the question "Would just using the first X bits of the random file be as representative of the file, as hashing the whole file into X bits of hash?" is this:

512 bits of hash are limited in what they can express. There have to eventually be hash collisions. While these are super unlikely ... how likely is it that a proper random source will generate the same first 512 bits of the file? Same, I believe.

A hashing algorithm is designed to produce as varying hashes as possible for only slightly different source data. This is useful for low entropy data, the most extreme everyday example of which is a human language text file. But in this case here, we're dealing with material that is as close to a hash in nature as it gets: Maximum entropy noise - just the thing that a hash is designed to be, except not random but derived from the source data. But if the source data is also max entropy, then it is also maximally unique, just like the hash.

Clarification edit:

I want to "uniquely" identify files of identical length, let's say 128 MB. Classically, hashing is used. But the files in this case are one-time pads, so they're "pure noise".

Does it even make sense to hash such a file? Or would the first few bytes (of the same length that the hash would have) serve the purpose on exactly the same level?

In other words, if a file's entropy is as high as can be, do the first 64 bytes represent the whole file as well as a 64 byte hash of the whole file would?

|improve this question
$\endgroup$
  • 2
    $\begingroup$ "-1"? If this question is dumb, I'd like to know why/how. Which is kind of the core of the question, anyway. Just downvoting it gets me nowhere. $\endgroup$ – Dreamspace President Feb 8 at 9:55
  • 1
    $\begingroup$ I don't understand your question, could you maybe include a simple example of what you mean and what the outcome might be? $\endgroup$ – AleksanderRas Feb 8 at 10:06
  • $\begingroup$ Added a clarification edit. $\endgroup$ – Dreamspace President Feb 8 at 10:17
  • 1
    $\begingroup$ Thanks for your efforts in making yourself clear, but I think the question is still a bit vague. What is exactly the setting? Could you describe in a self-contained manner the question you're considering? Something like "let $x$ by a $n$-bit string and let $y = h(x)$. What happens if we consider only the first $N$ entries of $y$?" Or something like that, depending on what is your question exactly. For example, I really don't get why you talk about hashes and one-time-pads simultanously. They're very different things! $\endgroup$ – Daniel Feb 8 at 10:22
  • $\begingroup$ I don't see how the question is unclear after the "Clarification edit", and I can't say it in the way you're asking for, because I don't speak math. I know that OTPs and hashes are very different things. I (hypothetically) have OTP files, and I want to uniquely identify those files. To uniquely identify files, we classically use hashes. But OTP files are "pure noise", and my thinking is: Does hashing such a file even make sense? Would not a portion of the file (Of the same length the hash would have.) serve the purpose on the exact same level? I do not understand how I could be more clear. $\endgroup$ – Dreamspace President Feb 8 at 10:30
2
$\begingroup$

Hashes still produce a "random" output, even if the input is a 128MB file of pure noise. Even if the 128MB file only differs by one bit then the hash changes completely, this is due to the avalanche effect.

Now from what I understand you're asking if a 512-bit hash (512 bits = 64 byte) of the first 512 bits of a 128MB file represents the whole file as well as a 512-bit hash of the whole file: The answer is no.

If two OTP files, let's say file $A$ and $B$, start with the same 512-bits (looking over the fact how unlikely that might be) and then start to differ you'd run into a problem:

Hash($A_{first 512 bits}$) = $H_A$, where $H_A$ is the hash-value of the first 512 bits of the file $A$.

Hash($B_{first 512 bits}$) = $H_B$, where $H_B$ is the hash-value of the first 512 bits of the file $B$.

Because the files $A$ & $B$ start with the same 512 bits you have hashed the same input and therefore also received the same hash-value: $H_A = H_B$.

If you hash both files (entirely instead of just the first 512 bits) then you would get a different hash value for the two files, because the two files are different from each other.

So the answer to your question if hashing OTPs makes sense is: Yes.

Feel free to comment for clarifications if something is unclear or if I've understood the question wrong.

|improve this answer
$\endgroup$
  • $\begingroup$ The logic of your answer is obvious. However, hashes collide, there's only so much you can say with 512 bits. The algorithm that created the OTP file would pack every byte to the brim with entropy. The collision fact makes me think that ultimately, the uniqueness power of both derived statements about the file would have the exact same weight. $\endgroup$ – Dreamspace President Feb 8 at 11:12
  • $\begingroup$ @DreamspacePresident Collisions will never, ever happen with 512-bit hashes (assuming that they hash function is not broken). You'll experience the death of the solar system before you ever encounter such a collision, even if you're actively looking for one. $\endgroup$ – Ella Rose♦ Feb 8 at 16:42
  • $\begingroup$ @EllaRose My point is that the likelihood, however low it may be, is just as high as encountering the same 512 bits of "perfect" randomness, so why bother hashing the whole file if just taking the first 512 bits will do? At least that's what I assume. $\endgroup$ – Dreamspace President Feb 8 at 17:21
  • $\begingroup$ As to why bother hashing the whole file versus just hashing the first 512 bits, it was not clear to me that is exactly what you were asking - you might include exactly that statement in your question. If you can assume your files are indeed uniformly random and cannot be tampered with, then the first 512 bits would serve as a valid uniqueness indicator This answer points out what happens if that assumption is unreliable. In general, you want to prepare for the worst-case scenario and use as few assumptions as possible. $\endgroup$ – Ella Rose♦ Feb 8 at 17:34
  • 1
    $\begingroup$ The chance of a 512-bit hash colliding is similar to the chance of two randomly generated files having their first 512 bits be the same. Both of these are equally practically too rare to consider. $\endgroup$ – Macil Feb 8 at 23:44

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.