2
$\begingroup$

I am using the NIST SP 800-22 to evaluate the performance of the Random Number Generator.

NIST SP 800-22 has a total of 15 tests. In each test, it has a recommendation for the size of the tested sequence. For example, in the Discrete Fourier Transform (Spectral) Test : "It is recommended that each sequence to be tested consist of a minimum of 1000 bits (i.e., n ≥ 1000)." So it means, the size of each sequence to be tested must be greater than 1000.

  • I selected n = 1,000,000 bits and the number of bitstream = 100. After I ran the Discrete Fourier Transform (Spectral) Test, I get a failed result (proportion: 95/100).

  • However, when I selected n = 100,000 bits and the number of bitstream = 100. After I ran the Discrete Fourier Transform (Spectral) Test, I get a passed result ( proportion: 96/100).

So, which value of n should I select? In order to confirm that the Random Number Generator passes the Discrete Fourier Transform (Spectral) Test, does the Random Number Generator always pass with all values of n ≥ 1000

$\endgroup$
8
  • $\begingroup$ Sorry, can you explain in more detail your question? $\endgroup$
    – Trần An
    Apr 13, 2022 at 11:46
  • $\begingroup$ You can't definitively say much about a failing test of an unknown RNG. Perform the test on /dev/urandom and see if it's the test parameters or it's NIST implementation. They're not good coders. What is your RNG exactly? $\endgroup$
    – Paul Uszak
    Apr 13, 2022 at 11:50
  • $\begingroup$ I think NIST SP800-22 does not care about which RNG is used, it only cares about the random bit sequence which is input for the NIST SP800-22 program. My RNG is TRNG $\endgroup$
    – Trần An
    Apr 13, 2022 at 11:59
  • $\begingroup$ The point I'm making is that if the RND is no good (bad), then the test should always fail. Otherwise why are you testing it? URAND fails these tests quite badly. Is it one of the inbuilt ones, or your own? $\endgroup$
    – Paul Uszak
    Apr 13, 2022 at 12:03
  • $\begingroup$ I design the TRNG, and I want to evaluate its performance. I saw NIST SP is the program can do this, so I generated the data from my TRNG, and run the NIST SP to check the performace $\endgroup$
    – Trần An
    Apr 13, 2022 at 12:07

1 Answer 1

0
$\begingroup$

1,250 bits x 10 streams works:-

    ------------------------------------------------------------------------------
RESULTS FOR THE UNIFORMITY OF P-VALUES AND THE PROPORTION OF PASSING SEQUENCES
------------------------------------------------------------------------------
   generator is <Blum-Blum-Shub>
------------------------------------------------------------------------------
 C1  C2  C3  C4  C5  C6  C7  C8  C9 C10  P-VALUE  PROPORTION  STATISTICAL TEST
------------------------------------------------------------------------------
  0   0   2   2   2   1   1   2   0   0  0.534146     10/10      FFT


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

as the above is a very good pass.

Assuming that your RNG is good, just running the FFT is inconclusive. You often get isolated test failures as randomness is pesky, especially as the sample size gets smaller. Randomness is very much a function of sample size. That's why test 'suites' have a broad variety of very simple and very complex tests. So no to "does the Random Number Generator always passes(sic) with all values of n ≥ 1000".

I would also be wary of 100 bitstreams. From experience again, 10 seems a good amount and allows a decent p value to be determined. You'll bang up against diminishing returns after >> 10 bitstreams. So do 10 bitstreams of 1,000,000 bits each (1.25 MB total) to run all the tests. You won't get any p values for the RandomExcursions(Variant) tests but you'll get an indication of success from the pass rate.

A rule of thumb in statistical circles is that you should take 30 samples✝ to be sure of distinguishing a distribution. So you'd use 30 bit streams of 1,000,000 bits each (3.75 MB). I've done that and SP 800-22 produces p values in the RandomExcursions(Variant) tests as well as pass/fail.

But before anything, if you've designed your own TRNG (and presumably the randomness extractor), run ent✝✝. It's ostensibly the most robust randomness test out there. Use 500 kB. If your samples fail ent, then your TRNG is bad.


✝ Whilst this is an Old Timer's rule of thumb, there is some debate as to its veracity these days with the advent of computers.

✝✝ Or now, there is an updated version called ent3000. It's specifically targetted at DIY TRNG Makers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.