[Dieharder-devel] more testing

Chris Doty-Humphrey cdhorz at gmail.com
Tue Apr 17 17:22:39 EDT 2012


Part way through this round of testing I noticed that diehard_runs was
flunking RNGs with no relationship to their quality.  Testing confirms
that it will fail even known good RNGs.  Here's an example output from
it on HC-256, an RNG that generates flawless output:
        diehard_runs| 15|   0|    100000|   13000|0.00000082|  FAILED
        diehard_runs| 15|   0|    100000|   13000|0.00000047|  FAILED
The command line that produced that was:
dieharder -g 200 -D 504 -D 16384 -d 15 -m 130
It looks like that has roughly a flat 8% chance of flunking an RNG
regardless of RNG quality.  So I went ahead and removed all the
diehard_runs failures from this set of data.  And you should ignore
the diehard_runs failure on simpleC from my previous post (though the
dab_filltree2 failure on that RNG remains valid).

Anyway, the overall picture from the data so far is: after
dab_monobit2 and dab_dct, the next best tests are dab_bytedistrib,
diehard_opso, diehard_oqso, dab_filltree2, and diehard_count_1s_str.
The only ones that pick up any RNGs missed by dab_monobit2 and dab_dct
so far are dab_bytedistrib (old_mwlac16 & clcg52_16) and dab_filltree2
(simpleC).  Though the tests that can take a variable amount of data
may be operating at a disadvantage since the times were set here by
adjusting the psamples on these tests, but by adjusting the amount of
data on dab_monobit2 and dab_dct from my previous set of runs.

Some non-LCG RNGs that use multiplication:

binarymult16 - fails 1 test: diehard_opso
garthy16 - fails 1 test: diehard_count_1s_str
garthy32 - no failures
rxmult16 - no failures
multish3x32 - no failures
multish4x16 - no failures
old_mwlac16 - fails 1 test: dab_bytedistrib

Some LCGs:

lcg32 - fails 11 tests: diehard_rank_6x8, diehard_bitstream,
diehard_opso, diehard_oqso, diehard_count_1s_str,
diehard_count_1s_byt, diehard_2dsphere, sts_monobit, sts_serial,
dab_bytedistrib, dab_filltree2
lcg32_extended - fails 11 tests: diehard_rank_6x8, diehard_bitstream,
diehard_opso, diehard_oqso, diehard_count_1s_str,
diehard_count_1s_byt, diehard_2dsphere, sts_monobit, sts_serial,
dab_bytedistrib, dab_filltree2
lcg32_16 - fails 5 tests: diehard_rank_6x8, diehard_opso,
diehard_oqso, diehard_2dsphere, and dab_bytedistrib
lcg36_16 - fails 3 tests: diehard_opso, diehard_oqso, dab_bytedistrib
lcg40_16 - fails 1 test: diehard_oqso
lcg44_16 - no failures
lcg48_16 - no failures

Some xor-based LCGs:

xlcg32_16 - fails 5 tests: diehard_rank_6x8, diehard_opso,
diehard_oqso, dab_bytedistrib, dab_filltree2
xlcg36_16 - fails 2 tests: diehard_opso, dab_bytedistrib
xlcg40_16 - no failures
xlcg44_16 - no failures
xlcg48_16 - no failures

Some combined LCGs & combined xor-based LCGs:

clcg48_16 - failed 4 tests: diehard_rank_6x8, diehard_oqso,
sts_serial, dab_bytedistrib
clcg52_16 - failed 1 test: dab_bytedistrib
clcg56_16 - no failures
clcg60_16 - no failures
clcg64_32 - failed 1 test: dab_bytedistrib
clcg68_32 - no failures
clcg72_32 - no failures
clcg76_32 - no failures
clcg80_32 - no failures
cxlcg64_32 - failed 1 test: dab_bytedistrib
cxlcg68_32 - no failures
cxlcg72_32 - no failures
cxlcg76_32 - no failures
cxlcg80_32 - no failures
cxlcg48_16 - failed 2 tests: diehard_opso, dab_bytedistrib
cxlcg52_16 - no failures
cxlcg56_16 - no failures
cxlcg60_16 - no failures

Some cyclic-buffer based RNGs:

mm32 - no failures
mm32_16 - no failures
mm32_awc - no failures
ranrot32 - no failures
cbuf_accum - no failures
dual_cbuf_accum - no failures
fibmul32_16 - no failures


More information about the Dieharder-devel mailing list