A cryptographic algorithm where the operations for encryption and decryption are identical is called a reciprocal cipher.
As dfaranha mentioned, this is mathematically an involution.
Most mechanical cipher machines use a reciprocal cipher, so it wouldn't need a seperate "encode mode" and "decode mode".
Some of the more famous reciprocal ciphers are:
- The Enigma machine was a reciprocal cipher machine
- Beaufort cipher
- Kama-sutra ciphers
- XOR-based stream ciphers
- One Time Pads (both XOR and Beaufort cipher versions)
Many modern single-block ciphers are built with a Feistel network structure that, in each round, cascades a reciprocal permutation P ("swap the left and right half of the block") and a keyed reciprocal substitution cipher S, to build a non-reciprocal bijective encryption function E(x) == P(S(X)) != D(X) = S(P(X). It is widely believed that practically any non-linear function can give adequate security if it is iterated over enough rounds.
While the overall cipher is not reciprocal, the parts that are reciprocal allow such ciphers to share parts between the encryption and decryption, so implementations use significantly less space (less code or less hard-wired ASIC area) than they would if they implemented encryption and decryption completely independently.
There are variants of the one-time pad that are not reciprocal (for example, using the Vigenère algorithm), but they are no more secure.