1
$\begingroup$

I had an idea of creating a pseudo-random binary sequence with that code:

import random as rd
p=[]
k=10**6
for i in range(k):
  p.append(rd.getrandbits(1))

s=''
for i in p:
  s+=str(i)
a=p.count(0)
b=p.count(1)
f = open('s.txt', 'w')
f.write(s)
f.close()

And on the stage of testing it with NIST tests there came a question of choosing right parameters for launching them. If I set the length of the sequence for this file for 100000 and number of bitstreams for 1, will it mean that from this file i will have only first 100000 bits tested as one sequence? Should the data in file be separated in some way?

I have also checked this question on crypto.stackexchange, but to be honest it didn't help me. Reading the official docs for NIST package didn't bring no understanding of bitstreams parameter too.

I will be very grateful if you explain me, how to set number of bitstreams parameter correctly and explain the meaning of it in this situation and the way the input data should be given to the tests.

$\endgroup$
  • 1
    $\begingroup$ What are you hoping to do by running the NIST tests? We know a priori that Python's random module does not make a good PRNG, because it uses the Mersenne twister, but the NIST tests won't tell you that—the NIST tests were written without knowledge of how Python's random module works. $\endgroup$ – Squeamish Ossifrage Oct 28 at 21:31
-1
$\begingroup$

Yes, the NIST tests are a little opaque. You'll find that the input file size should be (no. bitstreams) * (no. bits for testing). That means the input file will be automatically partitioned into (no. bitstreams) fragments, and the test suite run that times. Yo don't have to do anything. That creates the output histogram and the final P value assessments. It's typical to pick 10 streams.

So an input file should be at least 1MB in size. A good rule of thumb is 1 Byte * (no of bits for testing). But you won't get any meaningful P values for RandomExcursions and RandomExcursionsVariant until you go to a 10MB input as in ./assess 1000000. However, these two excursion tests don't generate P values until the input is >100MB.


Note1. The above input sizes are for binary files as I find them easier to manipulate. You can modify accordingly for ASCII representations. The tests work the same way.

Note2. Given that you're being Pythonic, the generator is the Mersenne Twister. It will pass with flying colours, even though academically it has some weaknesses in the very higher dimensions. If it doesn't, it's a code problem.

$\endgroup$
  • $\begingroup$ Thank you so much for your answer. And as I got from your message and according to this paper(table on page 5) I can be patient about getting P-values for Excursions tests with bitstreams longer than 10^6 bits , am I right? $\endgroup$ – R Georgy Oct 28 at 16:54
  • $\begingroup$ @RGeorgy Yes I think I'd concur. It's no secret that I'm a harsh critic of NIST randomness code, but I have a feeling in my water that these tests are not exactly 100% stable/reliable. I seem to remember not getting excursion Ps for 200MB files. Yet in refreshing my memory for this answer, I got Ps with files ~10MB. But I also got a 0/10 for LinearComplexity over /dev/urandom output, and that's very curious as it's very highly improbable... $\endgroup$ – Paul Uszak Oct 28 at 17:13
  • 1
    $\begingroup$ Wow, that's really unbelievable (about urandom testing) because I got quite good results for LCG provided with the package. Hope that the results of the tests will finally be right with long sequences.(for 2*10^8 bits and 10 bitstreams). Maybe will take dieharder package into consideration. $\endgroup$ – R Georgy Oct 28 at 17:25

Not the answer you're looking for? Browse other questions tagged or ask your own question.