# Judgment of Results from NIST Randomness Testing

I am using the NIST suite to test the randomness of binary numbers and I have a 64KB dataset. I am confused with the results in finalAnalysisReport.txt.

I have the following questions:

1. does the test pass only if there is no "*" in both "p-value" and "proportion" columns? example:
12 5 7 3 5 4 5 3 4 2 0.108791 45/50 * NonOverlappingTemplate
1. Does the "p-value" only need to be greater than 0.0001 to pass?
2. If I can't set the same parameters so that all tests pass, is it allowed to run 15 separate tests?
• Why such a small dataset? NIST runs better on larger datasets! Dec 28, 2023 at 22:38
• @ Maarten Bodewes♦ Thank you! I understand, but I'm limited by the size of the SRAM, do you have any other way to expand the true random number? Like writing a code or something? Dec 29, 2023 at 0:12
• If you can export it you may be able to stream it, put it into EEPROM etc.. You could also call the RNG multiple times, but I don't know if that will influence the state of the RNG; it depends on what you are trying to test in that case. I'm not familiar with your runtime environment. Dec 29, 2023 at 0:16

1.) The pass/fail threshold is printed at the bottom of the finalAnalysisReport.txt file. It looks like:-

The minimum pass rate for each statistical test with the exception of the random excursion (variant) test is approximately = x for a sample size = y binary sequences.

45/50 is a fail.

2.) No. $$\alpha = 0.01$$ is for the individual NIST tests. That seems reasonable given the relatively huge sample sizes. But the final $$\chi^2$$ test seems completely unreasonable with $$\alpha = 0.0001$$. It's almost as if NIST are encouraging poor generators to pass whereas common statistical analyses would use an $$\alpha = 0.05$$. Cui bono?

But. I am somewhat doubtful of the NIST tests. NIST/NSA objectives are not your objectives. And NIST are not great coders. I've just gotten:-

10   0   0   0   0   0   0   0   0   0  0.000000 *    0/10   *  Universal


for a 1 MB test from /dev/urandom. Zero point zero?

3ish.) 64 kB is much too low for a reliable NIST SPS test. So I suggest you generate more entropy or use other test suites more appropriate for smaller samples. There is ent and it's contemporary replacement ent3000.