2
$\begingroup$

I have three binary files:

  1. 31744 byte length,
  2. 5712980 byte length and
  3. 10806008 byte length.

The alphabet is all 256 symbols from 00 to ff. I compute the Shannon entropy, replacing probability with frequency.

  1. H = 4.291334237835409

  2. H = 7.999155088532762

  3. H = 7.993862849926811

Then I encrypted all three files with AES. Alphabet is still the same. I computed the Shannon entropy.

  1. H = 7.255697988479884

  2. H = 7.999970021706294

  3. H = 7.999829860973214

Ok, I see the difference and the entropy of encrypted data is higher, it is expected.

But why entropy of encrypted data close to 8 and doesn't grow up? Entropy of plaintext bytecode is growing up with the size of data, but entropy of encrypted bytecode doesn't grow up with the size of data.

$\endgroup$
6
  • 1
    $\begingroup$ "I computed the shannon entropy"; unlikely; Shannon entropy is defined entirely in terms of probability distribution; what probability distribution did you use? $\endgroup$
    – poncho
    Mar 15, 2016 at 21:46
  • 1
    $\begingroup$ @poncho, "replacing probability with frequency" $\endgroup$ Mar 15, 2016 at 21:53
  • 2
    $\begingroup$ So, what you did is effectively modelled the process of selecting a random byte from the file, and computed the entropy that would be in that byte? You can do that (and, since you have the exact probability distribution, you could compute the Shannon entropy). However that would also answer your question - such a process would give you an 8 bit value; of course the entropy in that byte can't go beyond 8. $\endgroup$
    – poncho
    Mar 15, 2016 at 21:59
  • $\begingroup$ @poncho, thanks! how I couldn't see that! I just need to enlarge alphabet. $\endgroup$ Mar 15, 2016 at 22:11
  • $\begingroup$ @poncho This was intended for you : How come the entropy of fixed files 1, 2 and 3 isn't 0? $\endgroup$
    – Paul Uszak
    Jan 8, 2023 at 16:58

2 Answers 2

4
$\begingroup$

What you did in your entropy estimation is effectively modeled the process of selecting a random byte from the file, and computed the entropy that would be in that byte? You can do that (and, since you have the exact probability distribution, you could compute the Shannon entropy). However that would also answer your question - such a process would give you an 8 bit value; of course the entropy in that byte can't go beyond 8.

(The comment converted into an answer because, well, it appears to be the answer)

$\endgroup$
0
$\begingroup$

If you have a text of any size that can consist of 256 different letters, then 8 is the max. possible Shannon entropy. You encrypted 3 files with AES and AES could improve the entropy of the first file to 7.25 - a not very impressive value however. The entropy of the other 2 files was already higher than AES's capabilities. At least it did not make emtropy worse. It's a little bit higher but it could have as well be less. That's just random. That's why it stayed the "same", but AES cannot make it much better...

$\endgroup$
1
  • $\begingroup$ Welcome, but... $H(E_k(1))$ is incorrect. A 32kB AES ciphertext has a Shannon entropy of $\approx$ 7.994 bits/byte. I don't know what the OP did to get such a low value, but it's wrong. $\endgroup$
    – Paul Uszak
    Jan 7, 2023 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.