0
$\begingroup$

Can anyone help me understand key entropy the mathematics behind it and the uses for it?

I understand most of it but I just want to understand what does $\log 2$ do like if I do. $\log 128 / \log 2 = 7$?

  • The Professor has really confused me here on this one on what the $\log 2$ mean? formula;

    $Entropy = \log(Phrases)/\log 2$

  • The professor gave this formula, too: How many bits can represent $X$ phases? Just take the $\log X$, and divide by $\log 2$.

$\endgroup$
2
  • $\begingroup$ Or written as $Entropy=log_2 (Phrases)$ this $\endgroup$
    – kelalaka
    Nov 4, 2018 at 20:30
  • 4
    $\begingroup$ FYI: adding things like "I need this in two days" etc can only have a negative impact on your audience. You make yourself look as someone that uses readers just as a tool. Avoid that in future questions. $\endgroup$
    – Bakuriu
    Nov 4, 2018 at 21:58

1 Answer 1

2
$\begingroup$

The Professor has really confused me here on this one on what the $\log 2$ mean? formula: $\text{Entropy} = \log(\text{Phrases})/\log 2$

If I understand the problem correctly, you are asking what the $\log 2$ is doing there in the denominator. This is essentially to ensure that the base of the logarithm (whether it's $10$ or $e$ or $2$) doesn't matter and you always get the computation result as a base-2 logarithm.

As a quick reminder: The base in a logarithm is the $b$ for which you are looking to find the $x$ such that $b^x=a$ for $x=\log a$.


How many bits [are needed to] represent $x$ [phrases]?

So you have $x$ values and you want to know how many bits you need to have in order to identify all of them. Well, first note that 1 bit can address 2 values, 2 bits can address 4 values, 3 bits can address 8 values, etc. $n$ bits can address $2^n$ values. Thus we are looking for the smallest value $n$ such that $x\leq 2^n$. So we first compute a logarithm, to an arbitrary base, on both sides of the inequality, which yields $\log x\leq \log(2^n)=n\cdot \log 2\Leftrightarrow \log x/\log 2\leq n$, thus $\log x/\log 2$ bits are sufficient.

$\endgroup$
1
  • $\begingroup$ Thanks for helping, I the professor gave this formula: How many bits can represent X phases? Just take the Log(X), and divide by Log (2). $\endgroup$ Nov 4, 2018 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.