I want to understand more about substitution-permutation network, if we modify it instead of carrying out the key-mixing, substitution, and permutation steps in alternating order for r rounds, the cipher instead first applies r rounds of key-mixing, then carries out r rounds of substitution, and finally applies r permutations. How this could affect the security, and what is the security of this construction?
Assuming your key mixing layer uses XOR:
- $M \oplus K_0 \oplus K_1 \oplus ... K_r$
- $K_0 \oplus K_1 \oplus ... K_r$ effectively compresses into one, single key
- Equivalent to $M ^ K$ in terms of security
More importantly, since the only key addition layer is at the front, and the S-box and permutation layer are assumed to be publicly known, a single known plaintext attack could break the construction: Simply encrypt the known block of information and then invert the permutation and S-box layers on the ciphertext. This leaves us with $M \oplus K$, which, since we know $M$, means we can do $M \oplus M \oplus K$ and recover $K$. Granted, $K$ is only the XOR sum of $K_0 \oplus K_1 \oplus ... K_r$, but we don't actually need the values of $K_0, K_1, ... K_r$ individually. We only need their sum in order to perform the encryption/decryption operation.
It is important for there to be key addition layers before and after the application of a round function, if the round function is to provide security. They do not have to be immediately prior to or after each application, i.e. adding a key, iterating the core permutation a bunch of times, then adding a key again is a valid strategy as well.