In the "Telegraphic Code to Insure Privacy and Secrecy in the Transmission of Telegrams" from 1882, Frank Miller assigned a number to around 14,000 code words. Bankers would select an "irregular" series of such words and exchange them with a remote partner. Any messages would be lined up below the next unused words on the pad for encoding. When you lined up the key with the word, you added the numbers corresponding to each (mod 14,000, essentially, though he didn't use that term). For decoding, you used subtraction.
Vernam independently patented a version in 1919 that operated character by character, using Baudot codes and XOR (though he didn't call it XOR). In Vernam's example, the plaintext A ("++---" in Baudot), and the key B ("+--++"). The resulting ciphertext is "-+-++" (G). Likewise, for decoding, G XOR B returns A.
Both methods suffice, because they provide a reversible operation. XOR was appealing for Vernam (whatever he called it to himself) because it was easy to implement in relays, because it is it's own inverse.
Your one time pad could involve any reversible operation(s) you choose, and can operate on any atom of information you choose. Similar techniques were even adapted to secure telephony solutions around the time of World War II, like SIGSALY, which added random noise to a telephone conversation, then subtracted that same noise on the other end. (Noise cancelling headphones also exploit the fact that sound waves are susceptible to what basically amounts to simple arithmetic.)
Of course, as user7576 notes, if you're just using a short key, you're not in "one time pad" territory anymore. A one time pad is supposed to be a long list of random information. If you re-use any chunk of that key, then it's no longer a "one time use" system.
The closest thing to a one-time pad with a short key might be the stream ciphers that use a pseudo-random number generator with a short seed (key). Such ciphers use the (pseudo) random output from the PRNG to simulate a one-time pad (but it's not a true one time pad, because the PRNG can be susceptible to attack).