4
$\begingroup$

Suppose, I suspect that the probability of occurrence of a particular bit as $0$ is $.558$. Since, it is not $.5$, so this is possibly a bias. Now, my question is:

What is the minimum number of independent trials I need to confirm my result?

I mean, observing this scenario for, say 100 times, must not be convincing. There should be a theoretical result regarding this. It would be very helpful if someone links such papers while answering.

What I am looking for is: Suppose I suspect a possible bias of $0.5 \pm x$, $x \in (0, .5]$. Then, at least, say $2^{1/x}$ trials are need to confirm my suspicion.

Related: https://stats.stackexchange.com/questions/417981/calculate-the-number-of-trials-needed-to-achieve-certain-number-of-successes-at

$\endgroup$
  • 2
    $\begingroup$ Is this just confidence intervals for a binomial distribution? $\endgroup$ – bmm6o Nov 25 '14 at 21:30
  • $\begingroup$ @bmm6o I don't think so! Or, probably I am misunderstanding. Can you elaborate a bit? $\endgroup$ – xxx--- Nov 26 '14 at 10:22
  • $\begingroup$ @owlstead Sorry for late response. But I am not getting what exactly bmm6o and you are talking about. $\endgroup$ – xxx--- Nov 26 '14 at 10:23
  • $\begingroup$ @ moderators: Could you migrate the question to math.stackexchange ? $\endgroup$ – xxx--- Nov 18 at 11:33
4
$\begingroup$

I searched the web, and finally (I think) I got:

In page 47, 48 of this thesis, it is mentioned that -

To detect a bias of $p(1+q)$ (where $p$ is the probability for an Uniform Random event), we need roughly $\dfrac{1}{pq^2}$ samples.

$\endgroup$
  • 4
    $\begingroup$ That seems to be the correct order of magnitude. Statisticians have more precise evaluations, taking into account the confidence level that we want in the conclusions. We are talking about a Chi-squared test with one degree of freedom. There are also methods to decide dynamically if we can stop the test or continue, for given confidence levels or reaching the wrong conclusions. $\endgroup$ – fgrieu Nov 26 '14 at 15:11
  • 1
    $\begingroup$ There's good stuff @ en.wikipedia.org/wiki/Checking_whether_a_coin_is_fair. $\endgroup$ – Paul Uszak Nov 18 at 15:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.