I calculated the entropy for "variable", "property" and their concatenation "variableproperty" (with e.g. the Shannon Entropy) and got $2.75, 2.5$ and $3.33$ respectively. These are all entropy per byte, if I don't err. But why is the last value greater than the other two and not lying between them?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
The calculator you've linked to is calculating the Shannon entropy of the character frequency distribution of your text. That is, if you type in e.g.
In particular, you would get the exact same result if you typed in
Essentially, the entropy value returned by that calculator may be considered a measure of how many questions it would take, on average, to correctly guess a letter picked at random from the input text, using yes/no questions like "is the letter a vowel?" or "is it before N in the alphabet?", assuming that we know the text it has been picked from.
Based on this, it should be obvious why the input