Cryptography Stack Exchange is a question and answer site for software developers, mathematicians and others interested in cryptography. It only takes a minute to sign up.
Sign up to join this community
I needed to use a scale in a code to determine which sentence in a file has been encrypted by a single-character key.
The scale that i currently used is: +2 for every occurrence of a vowel or a space and +1 for other alphabets, numbers and some characters. The sentence with the maximum value using above scale helped me in finding the solution.
Is there some standard scale used by cryptographers based on character frequency?
You can compute a statistical distance measure between the observed letter frequencies in your candidate plaintext and the typical letter frequencies in English text.
There are a number of different ways to measure the similarity between two statistical distributions, and I'm not aware of any theoretical or empirical studies on which of them would be optimal for recognizing correctly decrypted plaintext. However, in practice, the specific choice of distance measure seems to be of secondary importance compared to things like the amount of ciphertext available, knowledge of the nature of the plaintext, and the choice of which statistical features of the plaintext (e.g. individual letter frequencies, pair frequencies, vowel/consonant frequencies, etc.) to measure.
Thus, I'd suggest starting with something simple, like the total variation distance, which can be calculated simply by taking the difference between the observed and the expected frequency of each letter and summing their absolute values together, or the $\chi^2$-divergence, for which you need to sum the squares of the differences between the observed and the expected frequencies, divided by the expected frequency.
The full formulas also involve some constant factors that only depend on the message length, but for simple comparison purposes, the raw sums are enough. The $\chi^2$ distance is particularly useful if you need to estimate the absolute likelihood of a message being generated according to some given (e.g. uniformly random) distribution, although this requires some additional steps that are not needed for simple ranking by relative likelihood.
