3
$\begingroup$

i'm new to information theory. I know the standard perfect secrecy definition: $$ P(M=m|C=c) = P(M=m) , $$ for all $m$ and $c$, respectively the message and the cypher.

Now, i can use the entropic definitions to conclude that if the above equation holds, also the following is true: $$ H(M|C)=H(M) , $$ right?

But is it true also that i can go from the entropic formulation to the single probability equality?

$\endgroup$

1 Answer 1

4
$\begingroup$

Yes it is.

The conditional entropy $H(Y|X)$ is equal to $H(Y)$ if and only if $X$ and $Y$ are independent random variables. Hence $H(M|C)=H(M)$ implies that $M$ and $C$ are independent random variables so that in particular $$\mathbb P(M=m\ \&\ C=c)=\mathbb P(M=m)\mathbb P(C=c)$$ for all $m$ and $c$, but by the definition of conditional probability, we then have $$\mathbb P(M=m|C=c)=\frac{\mathbb P(M=m\ \&\ C=c)}{\mathbb P(C=c)}=\mathbb P(M=m).$$

$\endgroup$
2
  • $\begingroup$ Thanks. Have you got any reference books to consult about this? $\endgroup$ Mar 6 at 1:41
  • $\begingroup$ @forgetfuled I'd try Cover and Thomas "Elements of Information Theory". The result on conditional entropy of independent variables follows form the result on mutual information of independent variables in chapter 2. $\endgroup$
    – Daniel S
    Mar 6 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.