When bruteforcing a password (e.g. the common attacks on DES), where you have ciphertext only, you need a way to assess whether a decrypted plaintext is the right one. I believe the EFF DES machine does this by checking if the chars are printable. Of course, this only works for ASCII files, not things like images.
I'd like to measure the entropy (observed 0th order byte level) and see if it's above a threshold that can be attributed to randomness.
- Is that a good method?
- Is there another well known method?
- Where can I find tables that indicate given a randomly generated message of size X, there's p probability that it's entropy would be < ? (These types of tables are used, for instance, in chi-squared testing)
In response to D.W:
I have 5 ciphertexts all of 1 KB and no information about them. I was able to decrypt one by dictionary attack; it is plaintext ASCII. I imagine the others are also. If that were the case, it's easy to automatically distinguish valid decrypts from nonvalid (check that all chars are printable, not < 0x20 or > 0x80). Of course, the others might not be ASCII files, but until I try, I have no way of knowing.
But, I'm interested in this method in general. For instance, when looking at a sample output of an unknown cryptosystem, it would be a good way to assess if there are any obvious statistical problems. Or, when reverse engineering binary applications, and looking at data, and trying to assess if it's binary data or encrypted. I also find the question mathematically of interest even aside from applications.