I am given that $S_1=(S_1,P_1)$ and $S_2=(S_2,P_2)$ are sources, where $S_1=\{s_1,...,s_n\}$, $P_1(x_i)=p_i$ and $S_2=\{y_1,...,s_m\}$, $P_2(y_j)=q_j$.

I have to find the entropy of $S_{\lambda}=(S_i \cup S_2, P)$, where $0 \le \lambda \le 1$, $P(x_i)=\lambda p_i$, and $P(y_j)=(1-\lambda) q_j$.

I was unable to find something that I could understand on the process for finding the entropy of the union of two sources.

I am thinking that it is $H(S_{\lambda})=H(S_1 \cup S_2+S_2 \cup S_2)=H(S_1 + S_2+S_2 + S_2)=\sum_{i=1}^n p_i log(\frac{1}{p_i})+3\sum_{j=1}^mq_jlog(\frac{1}{q_j})$

But I haven't seen this before and wanted to make sure. Also any hint on how to proceed would be greatly appreciated, thank you.


closed as unclear what you're asking by Squeamish Ossifrage, AleksanderRas, Maarten Bodewes Jun 20 at 21:57

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ I don't understand your notation. It's hard to follow some of the subscripts and indexing variables (what is $y_1, \dots, s_m$? what is $i$ in $S_\lambda = (S_i \cup S_2, P)$?), and it's hard to tell where you're talking about sets versus probability spaces. It sounds like maybe you have two probability distributions, but you're taking the union and/or sum of their supports and I'm lost at this point. $\endgroup$ – Squeamish Ossifrage Feb 6 '18 at 17:36