# Do the NIST tools on GitHub measure entropy reliably? [closed]

There is a supposed NIST tool called SP800-90B_EntropyAssessment on that Git thing. It's designed to be used in accordance with NIST's Recommendation for the Entropy Sources Used for Random Bit Generation. Essentially the tools measure entropy in datasets from entropic data sources. It's meant to output 0 - 8 bits/byte, (but be warned that the Python tests take many hours to complete).

I don't know if that Git repository is officially sanctioned by the US Department of Commerce, or if these tools have been written by hobbyists during their lunch breaks.

• The C++ IID tool doesn't compile at all (bzlib.h: No such file or directory)

• The C++ non-IID tool gives the included 1.2MB $\pi$ sequence an entropy of 0.08 bits/byte, which is based on a Compression Test Estimate. That's a tenth of what I measure it at, and it's hard to reconcile with the scientific consensus that the constant is perfectly random. The expected value should be ~1 bit/byte give or take the min-entropy variance.

• It's not clear as to whether their $\pi$ sequence is IID or non-IID as it's an odd file format and the definition of IID is vague regarding the degrees of a Markov sequence.

• For some combinations of test and data source, you can get entropy > 8 bits/byte which infers a very good entropy generator or poorly validated code (definitely missing assert statements).

Does anyone have experience of this tool? I've never read any TRNG paper that's ever mentioned using it to estimate their entropy. And I've read a lot of them. Perhaps I've misunderstood what the tool does. Can this tool be relied upon to measure entropy correctly and securely?

• @Owl Why does the question need to be re-opened? It has an accepted answer. Do you intend to post a competing answer? As for re-opening: If the community feels a question should be re-opened, then they can cast votes to cause it to become so. Closing questions works similarly. This was closed by a former mod, but the community can still vote to re-open it. On that note... 1/2 – Ella Rose Aug 25 '19 at 3:09
• @Owl I'd like to take this opportunity to mention that "question closing gestapo" is a slight against every user that participates in curating the community, and is tip-toeing very closely to the "not nice" side of things. If you really want this re-opened, then make positive, helpful contributions to the community until you have 3,000 rep, then vote for it. – Ella Rose Aug 25 '19 at 3:10
• @EllaRose closing relevant questions down isn't nice either. By the way, do you get points for closing down questions?? If you want to complain about my comments to whoever manages stack overflow, then feel free. This is an incredibly frustrating problem. – Owl Aug 27 '19 at 10:54
• @Owl Obviously, no, you do not get points for closing questions. That would create an incentive structure that motivates users to close as many questions as possible, rather than only those they feel are inappropriate. I don't have to complain to stackexchanges management; It is my job to keep crypto.se clean, focused, and friendly. This is the reason why I mentioned I mentioned "be nice" and asked for why the question should be re-opened. If there was a convincing argument as to why this should be re-opened, it could be done. I asked you for such arguments, but I have yet to receive any. – Ella Rose Aug 27 '19 at 14:20
• Anyways: Comments should not be used for extended discussion, and we're creating notifications in Paul's inbox by having this conversation here. If this is to continue, please use The Side Channel or meta rather than posting content that does not directly address the question/questioner here. – Ella Rose Aug 27 '19 at 14:22

The C++ IID tool doesn't compile at all (bzlib.h: No such file or directory)

From the code:

 #include "bzlib.h" // sudo apt-get install libbz2-dev


Authors are indicating to install the bzip2 development package from your package manager, since is not installed by default. Their command will work on Debian-based distros.

You just need to install the package.
For me worked installing sudo eopkg install bzip2-devel (i'm on Solus), but it will vary upon OS and distro. Search for the bzip2 development package.

• You've helped but made it more confusing :-) It compiles now. Thanks, but now I can't get a sensible h value for $\pi$ with either Python or C++ tools . Must just be me :-( – Paul Uszak Mar 31 '18 at 0:16
• You can still edit /add if you have further insights... – Paul Uszak Mar 31 '18 at 2:01