I am given that $S_1=(S_1,P_1)$ and $S_2=(S_2,P_2)$ are sources, where $S_1=\{s_1,...,s_n\}$, $P_1(x_i)=p_i$ and $S_2=\{y_1,...,s_m\}$, $P_2(y_j)=q_j$.

I have to find the entropy of $S_{\lambda}=(S_i \cup S_2, P)$, where $0 \le \lambda \le 1$, $P(x_i)=\lambda p_i$, and $P(y_j)=(1-\lambda) q_j$.

I was unable to find something that I could understand on the process for finding the entropy of the union of two sources.

I am thinking that it is $H(S_{\lambda})=H(S_1 \cup S_2+S_2 \cup S_2)=H(S_1 + S_2+S_2 + S_2)=\sum_{i=1}^n p_i log(\frac{1}{p_i})+3\sum_{j=1}^mq_jlog(\frac{1}{q_j})$

But I haven't seen this before and wanted to make sure. Also any hint on how to proceed would be greatly appreciated, thank you.

  • $\begingroup$ I don't understand your notation. It's hard to follow some of the subscripts and indexing variables (what is $y_1, \dots, s_m$? what is $i$ in $S_\lambda = (S_i \cup S_2, P)$?), and it's hard to tell where you're talking about sets versus probability spaces. It sounds like maybe you have two probability distributions, but you're taking the union and/or sum of their supports and I'm lost at this point. $\endgroup$ – Squeamish Ossifrage Feb 6 '18 at 17:36