Return to Question

Suppose a $1000$-bit key used in the one-time pad is not randomly and uniformly generated.

 

1. Suppose that the values of the first $5$ bits are $0$, and the other $995$ bits are randomly generated and uniformly distributed (each bit with value $0$ and $1$ with probability $0.5$), what is the entropy of the key?
2. Suppose that each bit of the key is randomly generated but with value $0$ with probability $0.54$. What is the entropy of the key?

I have no idea how to start the two questions above. For part 1, I use the entropy formula $$-\sum_{x \in X}{P(x) \log_2 P(x)}$$ but I don't know what should I let $X$ be. Can anyone guide me?

EDIT: Proof of Additivity of Shannon Entropy

Aim: If $X$ and $Y$ are independent random variables, then $H(X,Y) = H(X)+H(Y)$

Proof: Since $X$ and $Y$ are independent random variables, we have $$P(X=x)=\sum_{y \in Y}{P(X=x,Y=y)}$$ $$P(Y=y)=\sum_{x \in X}{P(X=x,Y=y)}$$

Question: Do the equations above require the independence of $X$ and $Y$? If they are not independent, are the equations still true?

Suppose a $1000$-bit key used in the one-time pad is not randomly and uniformly generated.

1. Suppose that the values of the first $5$ bits are $0$, and the other $995$ bits are randomly generated and uniformly distributed (each bit with value $0$ and $1$ with probability $0.5$), what is the entropy of the key?
2. Suppose that each bit of the key is randomly generated but with value $0$ with probability $0.54$. What is the entropy of the key?

I have no idea how to start the two questions above. For part 1, I use the entropy formula $$-\sum_{x \in X}{P(x) \log_2 P(x)}$$ but I don't know what should I let $X$ be. Can anyone guide me?

EDIT: Proof of Additivity of Shannon Entropy

Aim: If $X$ and $Y$ are independent random variables, then $H(X,Y) = H(X)+H(Y)$

Proof: Since $X$ and $Y$ are independent random variables, we have $$P(X=x)=\sum_{y \in Y}{P(X=x,Y=y)}$$ $$P(Y=y)=\sum_{x \in X}{P(X=x,Y=y)}$$

Question: Do the equations above require the independence of $X$ and $Y$? If they are not independent, are the equations still true?

Suppose a $1000$-bit key used in the one-time pad is not randomly and uniformly generated.

1. Suppose that the values of the first $5$ bits are $0$, and the other $995$ bits are randomly generated and uniformly distributed (each bit with value $0$ and $1$ with probability $0.5$), what is the entropy of the key?
2. Suppose that each bit of the key is randomly generated but with value $0$ with probability $0.54$. What is the entropy of the key?

I have no idea how to start the two questions above. For part 1, I use the entropy formula $$-\sum_{x \in X}{P(x) \log_2 P(x)}$$ but I don't know what should I let $X$ be. Can anyone guide me?

EDIT: Proof of Additivity of Shannon Entropy

Aim: If $X$ and $Y$ are independent random variables, then $H(X,Y) = H(X)+H(Y)$

Proof: Since $X$ and $Y$ are independent random variables, we have $$P(X=x)=\sum_{y \in Y}{P(X=x,Y=y)}$$ $$P(Y=y)=\sum_{x \in X}{P(X=x,Y=y)}$$

Question: Do the equations above require the independence of $X$ and $Y$? If they are not independent, are the equations still true?

Suppose a $1000$-bit key used in the one-time pad is not randomly and uniformly generated.

1. Suppose that the values of the first $5$ bits are $0$, and the other $995$ bits are randomly generated and uniformly distributed (each bit with value $0$ and $1$ with probability $0.5$), what is the entropy of the key?
2. Suppose that each bit of the key is randomly generated but with value $0$ with probability $0.54$. What is the entropy of the key?

I have no idea how to start the two questions above. For part 1, I use the entropy formula $$-\sum_{x \in X}{P(x) \log_2 P(x)}$$ but I don't know what should I let $X$ be. Can anyone guide me?

EDIT: Proof of Additivity of Shannon Entropy

Aim: If $X$ and $Y$ are independent random variables, then $H(X,Y) = H(X)+H(Y)$

Proof: Since $X$ and $Y$ are independent random variables, we have $$P(X=x)=\sum_{y \in Y}{P(X=x,Y=y)}$$ $$P(Y=y)=\sum_{x \in X}{P(X=x,Y=y)}$$

Question: Do the equations above require the independence of $X$ and $Y$? If they are not independent, are the equations still true?

Suppose a $1000$-bit key used in the one-time pad is not randomly and uniformly generated.

1. Suppose that the values of the first $5$ bits are $0$, and the other $995$ bits are randomly generated and uniformly distributed (each bit with value $0$ and $1$ with probability $0.5$), what is the entropy of the key?
2. Suppose that each bit of the key is randomly generated but with value $0$ with probability $0.54$. What is the entropy of the key?

I have no idea how to start the two questions above. For part 1, I use the entropy formula $$-\sum_{x \in X}{P(x) \log_2 P(x)}$$ but I don't know what should I let $X$ be. Can anyone guide me?

Suppose a $1000$-bit key used in the one-time pad is not randomly and uniformly generated.

1. Suppose that the values of the first $5$ bits are $0$, and the other $995$ bits are randomly generated and uniformly distributed (each bit with value $0$ and $1$ with probability $0.5$), what is the entropy of the key?
2. Suppose that each bit of the key is randomly generated but with value $0$ with probability $0.54$. What is the entropy of the key?

I have no idea how to start the two questions above. For part 1, I use the entropy formula $$-\sum_{x \in X}{P(x) \log_2 P(x)}$$ but I don't know what should I let $X$ be. Can anyone guide me?

EDIT: Proof of Additivity of Shannon Entropy

Aim: If $X$ and $Y$ are independent random variables, then $H(X,Y) = H(X)+H(Y)$

Proof: Since $X$ and $Y$ are independent random variables, we have $$P(X=x)=\sum_{y \in Y}{P(X=x,Y=y)}$$ $$P(Y=y)=\sum_{x \in X}{P(X=x,Y=y)}$$

Question: Do the equations above require the independence of $X$ and $Y$? If they are not independent, are the equations still true?