(Converted to answer from a comment.)
If pen and paper are permitted, one could probably carry out the RC4 algorithm fairly easily using 256 numbered pieces of paper (small post-it notes might be ideal, since they'd be harder to move by accident) arranged in a 16 by 16 grid (I'd suggest numbering the notes in hex for easier indexing), with two coins or something to keep track of the $i$ and $j$ indices. The algorithm itself is simple enough to memorize, too. To destroy the internal state, just shuffle the notes (and cut them up and burn them if you want to be sure).
The hard part would be key setup. The usual way of keying RC4 is not only laborious to do by hand (it's more or less equivalent to 256 encryption steps), but doesn't really shuffle the state all that well. The standard remedy for that is to discard the initial part of the output, which makes for even more work for our would-be computerless cryptographer. If you can safely carry around the state as a stack of notes, you can do it once and then just keep churning out more of the same keystream for each message, but if not, some alternative key setup mechanism would be highly recommended.
Also, there's always Solitaire, although it needs a deck of cards and has significantly worse biases than RC4. In fact, the design of Solitaire bears a strong resemblance to RC4, and was almost certainly inspired by it. Given the known weaknesses of Solitaire, if you want a hand cipher using playing cards it might be better to go with RC4-52 (i.e. standard RC4, only with 52 instead of 256 elements in the state array), although I don't know if anyone's done any serious cryptanalysis of that. (It's almost certainly weaker than normal RC4, but I'm not sure how much weaker. Probably still better than Solitaire, though.)