I've generated 100M of random numbers in range (0..255). These numbers fail Dieharder tasts (bitnum = 8). However, I can pass this test by combining four bytes into an int. Randomness of the set didn't change, so, something isn't correct with the test settings. What could cause it?

    def prep_input(file_name, number_of_iterations,numbit=32):
        f = open(file_name, "a")            
        max_rand = (-1 + 2 ** numbit)
        a = len(str(max_rand))
        for i in range(0, number_of_iterations):
            f.write(str(randint(0, max_rand)).rjust(a) + "\n")


With header it looks like

# Dieharder
type: d
count: 16180337
numbit: 8

dieharder -a -g 202 -f input.txt >>result.txt

Report on bytes: Set fails when bytes are used
Report on ints: Set passes tests when ints are used


  • I've tested up to 4Gb of data. This is close to 2^32 although ASCII format requires ~4 characters (4 bytes) per a byte of actual data.
  • When this file is converted to byte stream it passws several tests and then reaches the end of data. When concatinated it passes tests better than ASCII int version.
  • When Python random is fed as a binary to

    import time import sys import os from random import randint

    j=0 div = 2

    while(True): sys.stdout.write(chr(randint(0, 255))) # newFileBytes = [randint(0, 255)] # os.write(1, (''.join(chr(i) for i in newFileBytes)).encode('charmap')) sys.stdout.flush() j+=1

    if 0 == j%div :
        dh_file = open("progress.log", "a")
        div = div*2

With python to_terminal.py | dieharder -a -g 200 >>test_result.dhres I get 16G (17179869184) as the last record in the log and out of 75 tests only two tests are weak the rest is a pass. Python passes DH when supplied as bit stream Clearly, the problem follows 8 bit ASCII representation.

  • $\begingroup$ Randomness is fundamentally impossible to measure. All statistical tests simply estimate randomness in different ways. I don't think it's surprising that different statistical tests produce different results. (Concretely, the output does look like something may be wrong, though.) $\endgroup$
    – yyyyyyy
    Jan 20 '19 at 23:29
  • $\begingroup$ Standard PRNG, 90% increase in pass rate thanks to data aggregation? There must be something wrong with the DH run parameters. $\endgroup$
    – Stepan
    Jan 21 '19 at 1:18
  • $\begingroup$ Feel free to comment downvotes. $\endgroup$
    – Stepan
    Jan 21 '19 at 1:22
  • $\begingroup$ Run the basic ENT test across your bytified data. If it passes, you've done the Dieharder test wrong. If it fails ENT badly, you've converted the data wrong. Then take it from there... $\endgroup$
    – Paul Uszak
    Jan 21 '19 at 4:19
  • 1
    $\begingroup$ dieharder -a requires much more data (in the order of $2^{32}$ bytes), see citations from the manual there. Also, testing a would-be CS(P)RNG using Diehard is somewhat like testing a bathyscaphe in a pool: not detecting a failure allows to draw no operational conclusion. $\endgroup$
    – fgrieu
    Jan 21 '19 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.