I am attempting to manually step through the AES algorithm. I have reached the MixColumns step, and am working with the following values:
30 CA 96 24 32 92 09 23 3F CA 43 63 7D 5A F8 96
My output for the first "row" (multiplying the first row of the constant matrix by each of the four columns in the matrix shown above, which was the output from the ShiftRows step) is as follows:
B4 B2 97 D4
With the following logic:
60 X 96 X 3F X 7D = B4 194 x 1B6 X CA X 5A = B2 12C X 1B X 43 X F8 = 18C (MOD 11B) = 97 48 X 69 X 63 X 96 = D4
For the second cell of the second row, I have the following logic:
(01 X CA) X (02 X 92) X (03 X CA) X (01 X 5A), with
X used here as a symbol denoting the exclusive or operation. The above simplifies to
CA X 124 X 25E X 5A, which evaluates to
3EA is clearly too large to be a single output value for MixColumns, which outputs a byte array. Thus, we must reduce it, as 18C above, by taking it
MOD 11B. However,
3EA (MOD 11B) evaluates to
2F1, which is still too large.
What is the procedure to be used in further reducing
3EA to a valid output byte for the AES MixColumns function?