So I wanted to check the outcome of a hardware TRNG and see if its entropy is good enough. I downloaded the source files for statistical test suit from nist.gov website, compiled them on my macOS machine with clang and tried some sample data files as inputs to check that everything works correct before proceeding to the next step. However, when I used data/data.pi file as an input for STS, the very first result P-value was incorrect!
For a Frequency test it was 0.534146 while the paper from NIST corresponding to that suit stated that the Frequency P-value for pi should be 0.578211 ("Appendix B Empirical Results for Sample Data", page 107). Other results were also incorrect, no surprise. I double-checked the selected options, but they are the same: 10 streams for 100,000 bits and a total of 1,000,000 bits. I also checked configuration parameters in defs.h file, they all match the corresponding ones in the paper.
My first concern was that during compilation process there were some warnings from the compiler. It was mainly unhappy with some uses of 'fabs(x)' for integer-types variables instead of 'abs(x)' in some files. I changed the code but the resulting P-values were still the same.
Any idea what might be the source of the problem? Results for other sample files do not match too.