hill cipher encryption way 1x3 plaintext matrix

Question

Can someone help me with a Hill cipher?

When do I have to use:

1. 1x3 plain text matrix (p1, p2, p3) * 3x3 key matrix
2. 3x3 key matrix * 3x1 plain text matrix

Or they are both correct?

I tried to search the internet, but found nothing useful.

Answer 1

Both representations are essentially equivalent. If

$$ \begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix} = \begin{bmatrix} k_{11} & k_{12} & k_{13} \\ k_{21} & k_{22} & k_{23} \\ k_{31} & k_{32} & k_{33} \end{bmatrix} \cdot \begin{bmatrix} p_1 \\ p_2 \\ p_3 \end{bmatrix}, $$

then, equivalently

$$ \begin{bmatrix} p_1 & p_2 & p_3 \end{bmatrix} \cdot \begin{bmatrix} k_{11} & k_{21} & k_{31} \\ k_{12} & k_{22} & k_{32} \\ k_{13} & k_{23} & k_{33} \end{bmatrix} = \begin{bmatrix} c_1 & c_2 & c_3 \end{bmatrix}. $$

This is a special case of the general rule that

$$ c = K p \iff p^\top K^\top = c^\top, $$

where $M^\top$ denotes the transpose of the matrix $M$.

Answer 2

When you talk about 1x3 or 3x1, we do not know if you are talking about COLxROW or ROWxCOL. The typical way of representing a vector in a matrix is ROWxCOL terms, so 3x1 is what you are looking for

The matrix multiplication uses a single column with $N$ entries multiplied by an $N*N$ matrix. Since this is multiplication the order does not matter, $P*K$ is the same as $K*P$, where $P$ is your 3x1 plaintext and $K$ is the 3x3 key.