4
$\begingroup$

Suppose, I suspect that the probability of occurrence of a particular bit as $0$ is $.558$. Since, it is not $.5$, so this is possibly a bias. Now, my question is:

What is the minimum number of independent trials I need to confirm my result?

I mean, observing this scenario for, say 100 times, must not be convincing. There should be a theoretical result regarding this. It would be very helpful if someone links such papers while answering.

What I am looking for is: Suppose I suspect a possible bias of $0.5 \pm x$, $x \in (0, .5]$. Then, at least, say $2^{1/x}$ trials are need to confirm my suspicion.

Related: https://stats.stackexchange.com/questions/417981/calculate-the-number-of-trials-needed-to-achieve-certain-number-of-successes-at

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ Is this just confidence intervals for a binomial distribution? $\endgroup$
    – bmm6o
    Nov 25, 2014 at 21:30
  • $\begingroup$ @bmm6o I don't think so! Or, probably I am misunderstanding. Can you elaborate a bit? $\endgroup$
    – hola
    Nov 26, 2014 at 10:22
  • $\begingroup$ @owlstead Sorry for late response. But I am not getting what exactly bmm6o and you are talking about. $\endgroup$
    – hola
    Nov 26, 2014 at 10:23
  • $\begingroup$ @ moderators: Could you migrate the question to math.stackexchange ? $\endgroup$
    – hola
    Nov 18, 2019 at 11:33
  • $\begingroup$ There is no such thing as "@moderators", you should flag your question to gain attention. $\endgroup$
    – Maarten Bodewes ♦
    Dec 18, 2019 at 17:06

1 Answer 1

Reset to default
4
$\begingroup$

I searched the web, and finally (I think) I got:

In page 47, 48 of this thesis, it is mentioned that -

To detect a bias of $p(1+q)$ (where $p$ is the probability for an Uniform Random event), we need roughly $\dfrac{1}{pq^2}$ samples.

Improve this answer
$\endgroup$
2
  • 4
    $\begingroup$ That seems to be the correct order of magnitude. Statisticians have more precise evaluations, taking into account the confidence level that we want in the conclusions. We are talking about a Chi-squared test with one degree of freedom. There are also methods to decide dynamically if we can stop the test or continue, for given confidence levels or reaching the wrong conclusions. $\endgroup$
    – fgrieu ♦
    Nov 26, 2014 at 15:11
  • 3
    $\begingroup$ There's good stuff @ en.wikipedia.org/wiki/Checking_whether_a_coin_is_fair. $\endgroup$
    – Paul Uszak
    Nov 18, 2019 at 15:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.