Suppose you remove the key addition layer. What happens then?
(Mouse over to see the answer. Try figuring out yourself first!)
Denote AES without the key addition layer by $\operatorname{f}$. Suppose an adversary has $\operatorname{f}(x)$. They can recover $x$ by applying $\operatorname{f}^{-1}(x)$. If there is no secret information (the key), then anyone can perform the permutation/inverse permutation at will. You cannot have confidentiality without a secret decryption key.
Edit in response to modified question
If the key is only applied once, then given a ciphertext an adversary can apply the inverse permutation on the ciphertext until they end up with $m' \oplus k$. This allows them to recover $k$ via exclusive-or with $m'$, where $m'$ is the result of $f'(x)$ and $f'$ is the first steps of AES up to the key addition layer.