Why are encryption and decryption so similar?
Block ciphers are designed with reversibility in mind. Of course, to maintain data privacy, they are only reversible with the proper key. This is because a block cipher with a given key is a permutation. It is a bijection of all possible block-sized inputs to all possible block-sized outputs. This makes it possible to reverse. The exact permutation is selected by the key, so what ciphertext block a given input maps to depends on the key. They are designed with this in mind to keep decryption simple.
Take the following permutation of a set of four different elements, chosen at random:
$\pmatrix{0&1&2&3\\1&3&2&0}$
Because it is a bijection, there always exists an inverse:
$\pmatrix{0&1&2&3\\3&0&2&1}$
As you can see, the permutation maps $1$ to $3$, so the inverse permutation maps $3$ back to $1$. A block cipher does just this, except the permutation is not fixed. Rather, it is "chosen" by the key. Each key selects a different permutation from all possible permutations. Additionally, there are far more than four elements. AES encryption permutes a set of $2^{128}$ different elements, chosen by the key. Decryption simply gives you the inverse, allowing you to recover the plaintext. All the complex math behind block ciphers is necessary because it would be impossible to store every bijection ($2^{128}! \times 2^{128}$) in a table.
Is this a property of all symmetric ciphers?
Block ciphers are not the only symmetric ciphers. Stream ciphers do not need to be reversible, as all they do is generate a pseudorandom stream of data, called the keystream, which is XORed with the plaintext. Decryption does not require running the stream cipher in reverse, but generating the same keystream (by selecting the same key) and XORing it with the ciphertext.
It may be interesting to note that it's possible to turn a regular block cipher into a stream cipher by using a mode of operation like counter mode, or CTR, which generates the keystream by encrypting an ever-increasing counter. Decryption with CTR mode actually involves using the block cipher in the encryption direction, so it's possible to implement CTR (or any other mode that turns a block cipher into a stream cipher) without ever needing to implement decryption. See the following diagram from Wikipedia:
Is this a property of all symmetric permutations?
Also no! Although it's pretty common for block ciphers to be designed to make the inverse permutation easy to implement, not all permutations do this, even if they are technically reversible. This is common for public permutations. That is a permutation that is fixed and does not require knowledge of a secret key to invert. They are common in sponge constructions, which get their high security by ensuring a substantial portion of the input to the permutation is secret, alleviating the need for a key. An example of a public permutation is Gimli, which was designed only in the forward direction:
Like Keccak, Ascon [15], etc., we evaluate performance only in the forward direction, and we consider only forward modes; modes that also use the inverse permutation require extra hardware area and do not seem to offer any noticeable advantages.
In addition to being easier to implement, there is no need to cryptanalyze its inverse. While it may not be intuitive, the fact is that sometimes the inverse of a permutation is less secure than the permutation in the forward direction. This was the case for the FROG block cipher. It was an AES candidate during the NIST AES competition which did not make it past the first round. It displayed slower diffusion in the decryption direction (the inverse permutation for a given key) than in the encryption direction.