Can Whirlpool take advantage of AES-NI?

Question

Whirlpool uses a modified version of the AES / Rijndael algorithm.

Can Whirlpool take advantage of AES-NI instructions (or possibly other specialized constructs) on Intel and Intel compatible processors?

Or is the underlying AES algorithm modified too much for AES-NI to be compatible with the algorithm?

---

As an additional (optional) question: would it be feasible and secure to adjust Whirlpool to be able to use AES-NI if Whirlpool is not compatible?

Answer 1

It cannot. The AES-NI extensions add six new instructions used to accelerate AES. Whirlpool uses a Rijndael-based cipher as its round function. From a page on the hash, the cipher works as so:

The Whirlpool hash function

WHIRLPOOL uses Merkle-Damgård strengthening and the Miyaguchi-Preneel hashing scheme with a dedicated 512-bit block cipher called $W$. This consists of the following. The bit string to be hashed is padded with a '1'-bit, then with a sequence of '0'-bits, and finally with the original length (in the form of a 256-bit integer value), so that the length after padding is a multiple of 512 bits. The resulting message string is divided into a sequence of 512-bit blocks $m_1$, $m_2$, ... $m_t$ which is then used to generate a sequence of intermediate hash values $H_0$, $H_1$, $H_2$, ... $H_t$. By definition, $H_0$ is a string of 512 '0'-bits. To compute $H_i$, $W$ encrypts $m_i$ using $H_{i-1}$ as key, and XORs the resulting ciphertext with both $H_{i-1}$ and $m_i$. Finally, the WHIRLPOOL message digest is $H_t$.

This can be expressed mathematically where $\mu$ maps input to the matrix $M_{8 \times 8}[\operatorname{GF}(2^8)]$, $W$ is the 10-round, 512-bit Whirlpool block cipher, $IV$ is a string of 512 0-bits, $M$ is the full message with a length of $t$ blocks, $m_i$ is a 512-bit block from $M$ at position $i$, and $m_t$ contains up to 512-bits of Merkle-Damgård padding (a 1-bit, as many 0-bits as necessary to pad the block to an odd multiple of 256, and a 256-bit right-justified binary representation of the total length of $M$).

\begin{align} \eta_i &= \mu(m_i)\\ H_0 &= \mu(IV)\\ H_i &= W_{H_{i-1}}(\eta_i) \oplus H_{i-1} \oplus \eta_i\\ \operatorname{Whirlpool}(M) &\equiv \mu^{−1}(H_t) \end{align}

The $W$ cipher and the $\mu$ function are described in more detail in the Whirlpool paper.

The AES-NI extension

To see whether or not AES-NI can accelerate $W$, we have to look at the purposes of each instruction. Reading through the Intel whitepaper on AES-NI, the instructions do the following:

• AESENC performs a single round of encryption using the provided round key. It performs the ShiftRows, SubBytes, and MixColumns steps on the data. The MixColumns step involves using the $\operatorname{GF}(2^8)$ polynomial $x^8 + x^4 + x^3 + x + 1$, whereas Whirlpool uses $x^8 + x^4 + x^3 + x^2 + 1$. In addition, SubBytes uses the AES S-box which Whirlpool does not use.* The different polynomial alone makes this instruction unusable for Whirlpool.

• AESDEC is identical to the above, but uses the inverse steps (i.e. the steps InvShiftRows, InvSubBytes, and InvMixColumns). The Whirlpool block cipher does not involve decryption, making this instruction useless.

• AESENCLAST is the same as AESENC, but it is intended for the final round of encryption. The primary difference is that it does not perform the MixColumns step. Unfortunately, it still performs SubBytes which uses the AES S-box, not the Whirlpool S-box.

• AESDECLAST is identical to AESENCLAST, but runs the inverse transforms.

• AESIMC passes the round keys through the InvMixColumns transformation to make them suitable for decryption. This still involves the incompatible polynomial, so it would not work even if Whirlpool could benefit from accelerated decryption.

• AESKEYGENASSIST implements the AES key schedule. It takes the cipher key and generates the necessary round keys. Whirlpool does not use a dedicated key schedule. Each round function acts as the key schedule, creating the round keys for the next round.

Now, even if these instructions could be used for Whirlpool's block cipher, they only operate on 128-bit blocks (although apparently it is possible to do 256-bit blocks as well, according to section 2.1 of this paper). Whirlpool's block cipher uses 512-bit blocks. The instructions would not work.

In summary, the instructions for encrypting and decrypting rounds are completely incompatible with Whirlpool. The only other instruction performs the AES key schedule, which Whirlpool does not use. You may be able to modify Whirlpool to use the unmodified AES rounds, but then it would no longer be Whirlpool. If you do need a hardware-accelerated cryptographic hash however, you could use the new Intel SHA extensions. These are used to accelerate SHA-1 and SHA-256. Alternatively, you could use a dedicated hashing algorithm that is designed to be accelerated by AES-NI, as is shown in this paper.

^{* Just because a cipher uses a different S-box than AES does not automatically disqualify it from being accelerated by AES-NI's implementation of SubBytes, if the S-box is similar enough. This is the case for Camellia, for example.}