What may be causing conflicts in Whirlpool implementation versions?

Question

I see by layman articles (e.g. at Wikipedia) that the Whirlpool hash function has seemingly three current versions. That is to say, they are referenced as Whirlpool-0 (original reference code), Whirlpool-T, an optimised for hardware version which was changed to whirlpool (let’s call it “current”) due to a somewhat weakening in a diffusion matrix (according to article).

Now, I see some python implementations do indeed use that “current” version. However, I have been using librhash on Linux (i.e. rhash -W \$file) and I’m verifying things with OpenSSL (openssl dgst -whirlpool \$file).

OpenSSL and rHash are consistent with each other, but different from the current version. I also noticed that OpenSSL and rHash do not seem to correlate to either of the previous versions.

Why is this so? Has there been some undocumented update to the default Whirlpool s-boxes or something? Is there a version conflict, or has there been a Whirlpool update I may have missed?

Answer 1

There is no difference.

The wiki page you referred to contains examples of hashes for all three versions of Whirlpool. For string "The quick brown fox jumps over the lazy dog", the current version should produce the following hash:

    B97DE512E91E3828B40D2B0FDCE9CEB3C4A71F9BEA8D88E75C4FA854DF36725F
    D2B52EB6544EDCACD6F8BEDDFEA403CB55AE31F03AD62A5EF54E42EE82C3FB35

Let's create a file containing this string and test different implementations.

    $ openssl dgst -whirlpool quick.txt
    whirlpool(quick.txt)= b97de512e91e3828b40d2b0fdce9ceb3c4a71f9bea8d88e75c4fa854df36725fd2b52eb6544edcacd6f8beddfea403cb55ae31f03ad62a5ef54e42ee82c3fb35

    $ rhash -W quick.txt
    b97de512e91e3828b40d2b0fdce9ceb3c4a71f9bea8d88e75c4fa854df36725fd2b52eb6544edcacd6f8beddfea403cb55ae31f03ad62a5ef54e42ee82c3fb35  quick.txt

Both match the hash from the wiki. Let's test the python implementation using this script:

    import whirlpool

    wp = whirlpool.new("The quick brown fox jumps over the lazy dog")
    hashed_string = wp.hexdigest()

    print(hashed_string)

The output is:

    b97de512e91e3828b40d2b0fdce9ceb3c4a71f9bea8d88e75c4fa854df36725fd2b52eb6544edcacd6f8beddfea403cb55ae31f03ad62a5ef54e42ee82c3fb35

Let's test the reference code from the Whirlpool home page by inserting the following lines at the end of makeISOTestVectors() function:

    const char* quick = "The quick brown fox jumps over the lazy dog";
    NESSIEinit(&w);
    NESSIEadd(quick, 8*strlen(quick), &w);
    NESSIEfinalize(&w, digest);
    printf("The hash-code is the following 512-bit string.\n\n");
    display(digest, DIGESTBYTES); printf("\n\n");
    fflush(stdout);

The output is:

    B97DE512E91E3828 B40D2B0FDCE9CEB3 C4A71F9BEA8D88E7 5C4FA854DF36725F
    D2B52EB6544EDCAC D6F8BEDDFEA403CB 55AE31F03AD62A5E F54E42EE82C3FB35

All implementations produce the same hash. Where did you found a mismatching version?

Answer 2

Ok. I think I will attempt answering this myself. Given that (at least on linux), perl, openssl have gone down the same path as the rhash author (I am not sure who in fact, implemented this first), the reason for a different digest, is that, due to restricting the input message from $2^{512}$ bits to $2^{64}$ bits max, the first $512$ rows of $4 \times 32$-bit hex words of the S-boxes have been kept, and the last $512$ rows of the tables have been truncated.

I am not in the position of evaluating the security implications of this, and thus do not wish to accept this answer by myself yet (hoping for more professional input).

If however, solid user bases such as perl and openssl use it, I am inclined to suppose this is a viable implementation.

As a sub-question then, I still wonder why this has not been 'updated' as a 'version 4' or a $64$-bit input version more clearly?

EDIT: The "latter half" of S-boxes I thought were truncated are in fact, obsoleted ones from the reference code. This leads to my assumption being wrong and now for sure, it seems there is this version 4 out in the wild.

Surely, with an important hash as whirlpool, someone would have some insight into this. reference code asserts python code != {perl,openssl,rhash}!

To add to answer above, as I can't comment on it?

EDIT to Kerukuro: I can't comment?

on arch linux, (I also asked someone on another distro), I get :

cat quick.txt

The quick brown fox jumps over the lazy dog cat quick.txt | wc 1 9 44

openssl dgst -whirlpool quick.txt

whirlpool(quick.txt)= 72687676756b91ad986f2e56df761b354b748bc20098354b017b924e82cc67ae059da85f009d1a17c0f12ec0e644c0c3a193f3fc0fee22f053edbfcd95cbf873

same intel sandy bridge (arch 64 bit distro): rhash -W quick.txt

72687676756b91ad986f2e56df761b354b748bc20098354b017b924e82cc67ae059da85f009d1a17c0f12ec0e644c0c3a193f3fc0fee22f053edbfcd95cbf873 quick.txt

perl: whirlpoolsum quick.txt Possible precedence issue with control flow operator at /usr/bin/vendor_perl/whirlpoolsum line 227. 72687676756b91ad986f2e56df761b354b748bc20098354b017b924e82cc67ae059da85f009d1a17c0f12ec0e644c0c3a193f3fc0fee22f053edbfcd95cbf873 quick.txt

The other examples you mention produce the expected result (python, reference nessie source).

So, just to summarise: python does catch the newlines, and for rhash, openssl and echoing, one should do for correct hashing:

echo "The quick brown fox jumps over the lazy dog" | tr -d '\n' | openssl dgst -whirlpool