# What is entropy?

We discuss a lot of topics and use measures of entropy to determine how difficult it is for an attacker to be successful. What does entropy mean in the context of cryptography? How is entropy calculated in the general case?

• I think of entropy like this: Say the attacker wants to know your password and you are willing to tell her the password by answering her questions... What is the minimum number of questions with yes/no answer the attacker needs to ask in order to find the password? – Panos Kal. Jun 29 '16 at 16:01

To give you a short answer to your question: The common notion of entropy is the notion of Shannon entropy. The information content $H_x$ of a value $x$ that occurs with probability $\Pr[x]$ is $$H_x = -\log_2(\Pr[x]) \text.$$ The entropy of a random source is the expected information content of the symbol it outputs, that is $$H(X) = E[H_X] = \sum_x \Pr[x]H_x = \sum_x -\Pr[x]\log_2(\Pr[x]) \text.$$
You can interpret this as the expected uncertainty about a symbol $x$ knowing only the distribution according to which $x$ is chosen. For the case of bit strings the important message is, the entropy not always equals the bit length of $x$. I.e. if you want to seed a pseudo-random generator, it is important to choose a seed with high entropy. If you use the actual time as seed the entropy of the seed is very low, as everybody can easily guess parts of the seed (year, day, perhaps even hour and minute). Ask Netscape, they should have learned this…