# How many bits of entropy would I get from 300 coin tosses? [closed]

Bonus question, what is the formula to calculate this?

• If it's an unbiased coin, you get one bit per toss. Since Shannon entropy is additive for independent events, you get 300 bits total. (This assumes that you toss 300 times, not that you toss 300 indistinguishable coins together) – CodesInChaos Jan 5 '15 at 15:09
• The formula is $n$ bits of entropy for $n$ coin tosses with a fair coin. – Guut Boy Jan 5 '15 at 15:23
• @CodesInChaos Re: fair coin, does is really matter when each toss varies in rotational speed and length of time it rotates? – user21767 Jan 5 '15 at 15:33
• @user21767, "fair coin" refers to a coin that has an even chance of turning up heads or tails, one that shows no bias over multiple flips. It encompasses the entire process of flipping a coin, including rotations, speed, wind, gravity, whatever; and it is considered fair only if the outcome is statistically unpredictable. – John Deters Jan 11 '15 at 4:29

Let $X\colon\;\Omega\to\Sigma$ be a random source that emits symbols from a finite alphabet $\Sigma$. Write $p_\sigma:=\Pr[X=\sigma]$ for the probability that $\sigma$ is emitted. Then, the entropy $H(X)$ of $X$ is defined as $$H(X) = -\sum_{\sigma\in\Sigma}p_\sigma\log_2p_\sigma\text.$$ (If $p_\sigma$ is $0$, one uses the convention $p_\sigma\log_2p_\sigma=0$.)
In this case, $\Sigma=\{0,1\}^{300}$, hence $\lvert\Sigma\rvert=2^{300}$, and, assuming the coin is fair, all $p_\sigma$ are equal to $1/\lvert\Sigma\rvert=2^{-300}$. Therefore: $$H(X)=-\sum_{\sigma\in\Sigma}p_\sigma\log_2p_\sigma=-\lvert\Sigma\rvert\cdot2^{-300}(-300)=300\text,$$ which is just what one would intuitively expect from 300 random bits.
Note: One could also repeatedly apply the fact that, for independent random variables $X\colon\;\Omega\to\Sigma_1$ and $Y\colon\;\Omega\to\Sigma_2$, their combined entropy — that is, the entropy of $$X\times Y\colon\;\Omega^2\to\Sigma_1\times\Sigma_2,\;(\omega_1,\omega_2)\mapsto(X(\omega_1),Y(\omega_2))$$ in the product space — is $$H(X\times Y)=H(X)+H(Y)\text,$$ to $X\colon\;\Omega\to\{0,1\}$ uniformly distributed.