2
$\begingroup$

I'm working with the dieharder package to test a PRNG, but I don't quite understand the ntup parameter.

I get that the ntuple should be a set of consecutive bits. But why is the ntup parameter for most tests set to 0? Is it because these tests don't need ntup to operate? What would exactly change if one would change the ntup parameter, or at least what should be expected?

To show an example of the output I just copied output from another question. Can someone please give me a simple explanation what the ntup exactly does?

Sample dieharder output:

            dieharder version 3.31.1 Copyright 2003 Robert G. Brown          #
##  
   rng_name    |           filename             |rands/second|
     file_input|                        fish.txt|  7.26e+06  |
#   
        test_name   |ntup| tsamples |psamples|  p-value |Assessment
# b       #
   diehard_birthdays|   0|       100|     100|0.65344498|  PASSED     
      diehard_operm5|   0|   1000000|     100|0.11726095|  PASSED     
  diehard_rank_32x32|   0|     40000|     100|0.84203088|  PASSED     
    diehard_rank_6x8|   0|    100000|     100|0.73796762|  PASSED     
   diehard_bitstream|   0|   2097152|     100|0.38051493|  PASSED      
        diehard_opso|   0|   2097152|     100|0.04995365|  PASSED      
        diehard_oqso|   0|   2097152|     100|0.59990831|  PASSED      
         diehard_dna|   0|   2097152|     100|0.62220146|  PASSED      
diehard_count_1s_str|   0|    256000|     100|0.86262588|  PASSED       
diehard_count_1s_byt|   0|    256000|     100|0.12560712|  PASSED       
 diehard_parking_lot|   0|     12000|     100|0.28041223|  PASSED       
    diehard_2dsphere|   2|      8000|     100|0.76183470|  PASSED       
    diehard_3dsphere|   3|      4000|     100|0.17846568|  PASSED       
     diehard_squeeze|   0|    100000|     100|0.00384865|   WEAK        
        diehard_sums|   0|       100|     100|0.43470008|  PASSED       
        diehard_runs|   0|    100000|     100|0.70560801|  PASSED         
        diehard_runs|   0|    100000|     100|0.98961342|  PASSED       
       diehard_craps|   0|    200000|     100|0.18043071|  PASSED        
       diehard_craps|   0|    200000|     100|0.89290364|  PASSED       
 marsaglia_tsang_gcd|   0|  10000000|     100|0.00000000|  FAILED        
 marsaglia_tsang_gcd|   0|  10000000|     100|0.00000000|  FAILED       
         sts_monobit|   1|    100000|     100|0.72613955|  PASSED        
            sts_runs|   2|    100000|     100|0.26712378|  PASSED        
          sts_serial|   1|    100000|     100|0.94935591|  PASSED        
          sts_serial|   2|    100000|     100|0.93198856|  PASSED       
          sts_serial|   3|    100000|     100|0.99554380|   WEAK          
          sts_serial|   3|    100000|     100|0.72444970|  PASSED         
          sts_serial|   4|    100000|     100|0.19981717|  PASSED          
          sts_serial|   4|    100000|     100|0.06124493|  PASSED        
          sts_serial|   5|    100000|     100|0.87811046|  PASSED       
          sts_serial|   5|    100000|     100|0.09618787|  PASSED        
          sts_serial|   6|    100000|     100|0.87317579|  PASSED       
          sts_serial|   6|    100000|     100|0.99688012|   WEAK         
          sts_serial|   7|    100000|     100|0.50043383|  PASSED        
          sts_serial|   7|    100000|     100|0.61746527|  PASSED       
          sts_serial|   8|    100000|     100|0.97376805|  PASSED        
          sts_serial|   8|    100000|     100|0.94872352|  PASSED        
          sts_serial|   9|    100000|     100|0.35958803|  PASSED        
          sts_serial|   9|    100000|     100|0.39252366|  PASSED        
          sts_serial|  10|    100000|     100|0.40682407|  PASSED       
          sts_serial|  10|    100000|     100|0.69846273|  PASSED      
          sts_serial|  11|    100000|     100|0.14757958|  PASSED      
          sts_serial|  11|    100000|     100|0.85142983|  PASSED     
          sts_serial|  12|    100000|     100|0.33628714|  PASSED   
          sts_serial|  12|    100000|     100|0.80400201|  PASSED       
          sts_serial|  13|    100000|     100|0.50635238|  PASSED     
          sts_serial|  13|    100000|     100|0.04113439|  PASSED         
          sts_serial|  14|    100000|     100|0.48030593|  PASSED    
          sts_serial|  14|    100000|     100|0.83615004|  PASSED   
          sts_serial|  15|    100000|     100|0.85634237|  PASSED    
          sts_serial|  15|    100000|     100|0.86413582|  PASSED     
          sts_serial|  16|    100000|     100|0.81247787|  PASSED    
          sts_serial|  16|    100000|     100|0.62279344|  PASSED   
         rgb_bitdist|   1|    100000|     100|0.10521730|  PASSED    
         rgb_bitdist|   2|    100000|     100|0.36419006|  PASSED   
         rgb_bitdist|   3|    100000|     100|0.50848488|  PASSED   
         rgb_bitdist|   4|    100000|     100|0.91911028|  PASSED   
         rgb_bitdist|   5|    100000|     100|0.68355135|  PASSED   
         rgb_bitdist|   6|    100000|     100|0.07298074|  PASSED   
         rgb_bitdist|   7|    100000|     100|0.99703911|   WEAK    
         rgb_bitdist|   8|    100000|     100|0.50843903|  PASSED   
         rgb_bitdist|   9|    100000|     100|0.51151893|  PASSED   
         rgb_bitdist|  10|    100000|     100|0.40558627|  PASSED   
         rgb_bitdist|  11|    100000|     100|0.27640943|  PASSED   
         rgb_bitdist|  12|    100000|     100|0.64415540|  PASSED   
rgb_minimum_distance|   2|     10000|    1000|0.77930849|  PASSED   
rgb_minimum_distance|   3|     10000|    1000|0.89974521|  PASSED   
rgb_minimum_distance|   4|     10000|    1000|0.52655953|  PASSED   
rgb_minimum_distance|   5|     10000|    1000|0.01029230|  PASSED   
    rgb_permutations|   2|    100000|     100|0.49947588|  PASSED    
    rgb_permutations|   3|    100000|     100|0.48976917|  PASSED   
    rgb_permutations|   4|    100000|     100|0.99435735|  PASSED   
    rgb_permutations|   5|    100000|     100|0.18313717|  PASSED   
      rgb_lagged_sum|   0|   1000000|     100|0.11846988|  PASSED   
      rgb_lagged_sum|   1|   1000000|     100|0.20793259|  PASSED   
      rgb_lagged_sum|   2|   1000000|     100|0.08406645|  PASSED   
      rgb_lagged_sum|   3|   1000000|     100|0.00002903|   WEAK    
      rgb_lagged_sum|   4|   1000000|     100|0.00000000|  FAILED   
      rgb_lagged_sum|   5|   1000000|     100|0.00156593|   WEAK    
      rgb_lagged_sum|   6|   1000000|     100|0.13398055|  PASSED   
      rgb_lagged_sum|   7|   1000000|     100|0.00000071|  FAILED  
      rgb_lagged_sum|   8|   1000000|     100|0.07282751|  PASSED   
      rgb_lagged_sum|   9|   1000000|     100|0.00000653|   WEAK    
      rgb_lagged_sum|  10|   1000000|     100|0.24179580|  PASSED   
      rgb_lagged_sum|  11|   1000000|     100|0.00620851|  PASSED   
      rgb_lagged_sum|  12|   1000000|     100|0.24310357|  PASSED   
      rgb_lagged_sum|  13|   1000000|     100|0.15323532|  PASSED   
      rgb_lagged_sum|  14|   1000000|     100|0.00000174|   WEAK    
      rgb_lagged_sum|  15|   1000000|     100|0.00000000|  FAILED   
      rgb_lagged_sum|  16|   1000000|     100|0.39385345|  PASSED   
      rgb_lagged_sum|  17|   1000000|     100|0.11381934|  PASSED   
      rgb_lagged_sum|  18|   1000000|     100|0.78715021|  PASSED   
      rgb_lagged_sum|  19|   1000000|     100|0.00000000|  FAILED   
      rgb_lagged_sum|  20|   1000000|     100|0.52371093|  PASSED   
      rgb_lagged_sum|  21|   1000000|     100|0.00531931|  PASSED   
      rgb_lagged_sum|  22|   1000000|     100|0.19857791|  PASSED   
      rgb_lagged_sum|  23|   1000000|     100|0.00000000|  FAILED   
      rgb_lagged_sum|  24|   1000000|     100|0.00000431|   WEAK    
      rgb_lagged_sum|  25|   1000000|     100|0.02471464|  PASSED   
      rgb_lagged_sum|  26|   1000000|     100|0.07546643|  PASSED   
      rgb_lagged_sum|  27|   1000000|     100|0.01786362|  PASSED   
      rgb_lagged_sum|  28|   1000000|     100|0.28778610|  PASSED   
      rgb_lagged_sum|  23|   1000000|     100|0.00000000|  FAILED 
      rgb_lagged_sum|  30|   1000000|     100|0.13632978|  PASSED   
      rgb_lagged_sum|  31|   1000000|     100|0.00000000|  FAILED   
      rgb_lagged_sum|  32|   1000000|     100|0.11176539|  PASSED   
     rgb_kstest_test|   0|     10000|    1000|0.29113864|  PASSED   
     dab_bytedistrib|   0|  51200000|       1|0.00000000|  FAILED   
             dab_dct| 256|     50000|       1|0.93719430|  PASSED   
Preparing to run test 207.  ntuple = 0 
        dab_filltree|  32|  15000000|        1|0.60930301|  PASSED  
        dab_filltree|  32|  15000000|       1|0.29252232|  PASSED     
Preparing to run test 208.  ntuple = 0   
       dab_filltree2|   0|   5000000|       1|0.64246455|  PASSED     
       dab_filltree2|   1|   5000000|       1|0.20829118|  PASSED     
Preparing to run test 209.  ntuple = 0
        dab_monobit2|  12|  65000000|       1|1.00000000|  FAILED
Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

We can discern the following from the man page:-

  -n ntuple - set ntuple length for tests on short bit strings that permit
     the length to be varied (e.g. rgb bitdist).

Clearly you know what a tuple is in computer parlance. So it affects the sampling 'window' used by some of the tests. So that's it for that, but now a warning...

dieharder takes many optional parameters that alter its behavior. If you tweak it too much, how will you know whether the generator under test is bad, or whether your amendments to dieharder's behavior is what's causing failures? Many of those individual tests are based on certain statistical distributions like chi or exponential. The bit distribution tests are affected thusly:-

This test must be run with -n ntuple for ntuple > 0. Note that if ntuple 
> 12, one should probably increase tsamples so that each of the 2^ntuple bins 
should end up with an average of around 30 occurrences. Note also that the 
memory requirements and CPU time requirements will get quite large by e.g. 
ntuple = 20 -- use caution when sampling the distribution of very large 
ntuples.

That seems confusing and contradictory as max(ntuple) = 256 with tsamples = 50,000 only. It is unlikely that dieharder has been tested against all permutations of generator & parameter. Therefore I would have little confidence in the results of testing if dieharder is not running at default settings. If you review literature where dieharder is used, you will be hard pressed to find anyone who varies any parameters that affect the statistical tests.

I'd wouldn't change them.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.