There is a supposed NIST tool called SP800-90B_EntropyAssessment on that Git thing. It's designed to be used in accordance with NIST's Recommendation for the Entropy Sources Used for Random Bit Generation. Essentially the tools measure entropy in datasets from entropic data sources. It's meant to output 0 - 8 bits/byte, (but be warned that the Python tests take many hours to complete).

I don't know if that Git repository is officially sanctioned by the US Department of Commerce, or if these tools have been written by hobbyists during their lunch breaks.

  • The C++ IID tool doesn't compile at all (bzlib.h: No such file or directory)

  • The C++ non-IID tool gives the included 1.2MB $ \pi $ sequence an entropy of 0.08 bits/byte, which is based on a Compression Test Estimate. That's a tenth of what I measure it at, and it's hard to reconcile with the scientific consensus that the constant is perfectly random. The expected value should be ~1 bit/byte give or take the min-entropy variance.

  • It's not clear as to whether their $ \pi $ sequence is IID or non-IID as it's an odd file format and the definition of IID is vague regarding the degrees of a Markov sequence.

  • For some combinations of test and data source, you can get entropy > 8 bits/byte which infers a very good entropy generator or poorly validated code (definitely missing assert statements).

Does anyone have experience of this tool? I've never read any TRNG paper that's ever mentioned using it to estimate their entropy. And I've read a lot of them. Perhaps I've misunderstood what the tool does. Can this tool be relied upon to measure entropy correctly and securely?


closed as off-topic by e-sushi Mar 31 '18 at 4:45

  • This question does not appear to be about cryptography within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – Ella Rose Jul 10 at 15:08

The C++ IID tool doesn't compile at all (bzlib.h: No such file or directory)

From the code:

 #include "bzlib.h" // sudo apt-get install libbz2-dev

Authors are indicating to install the bzip2 development package from your package manager, since is not installed by default. Their command will work on Debian-based distros.

You just need to install the package.
For me worked installing sudo eopkg install bzip2-devel (i'm on Solus), but it will vary upon OS and distro. Search for the bzip2 development package.

  • $\begingroup$ You've helped but made it more confusing :-) It compiles now. Thanks, but now I can't get a sensible h value for $ \pi $ with either Python or C++ tools . Must just be me :-( $\endgroup$ – Paul Uszak Mar 31 '18 at 0:16
  • $\begingroup$ You can still edit /add if you have further insights... $\endgroup$ – Paul Uszak Mar 31 '18 at 2:01

Not the answer you're looking for? Browse other questions tagged or ask your own question.