I've just used the FIPS 140-2 tests as implemented in rngtools, the rngtest. I ran it on a 12.3MB file of pak8 compressed ASCII text. It passed most of the tests, with a 0.2% failure rate. Monobit and Continuous runs tests actually complete without any failures.
It seems very lax at detecting randomness. What's the point of this test if it (effectively) passes off a highly structured zip like file as good quality random numbers?
Here is the detail...
cat /tmp/raw.txt.fp8 | rngtest
rngtest: starting FIPS tests... rngtest: entropy source drained rngtest: bits received from input: 98327312 rngtest: FIPS 140-2 successes: 4906 rngtest: FIPS 140-2 failures: 10 rngtest: FIPS 140-2(2001-10-10) Monobit: 0 rngtest: FIPS 140-2(2001-10-10) Poker: 1 rngtest: FIPS 140-2(2001-10-10) Runs: 1 rngtest: FIPS 140-2(2001-10-10) Long run: 8 rngtest: FIPS 140-2(2001-10-10) Continuous run: 0