Ciphertext with uneven letter distribution

Question

I have a ciphertext encrypted with an unknown method. It is part of a challenge (personal quest), so I can not post it. The letter statistics is the following:

y : 10
w : 8
x : 8
t : 5
v : 5
e : 5
c : 5
d : 5
u : 4
z : 4
f : 3
b : 2

Because of the uneven letter distribution and repeating two letter sentences, I tried to approach as a monoalphabetic substitution cipher without luck. I tried manually and also with this python code.

The previous challenge was a Caesar cipher, so I do not expect any complicated method. Are there other encryption methods which produce such a letter distribution?

UPDATE: I am no more interested in this challenge. Here is the whole question, if someone is interested:

This is your first personal quest. Decipher it in order to access Level Two! But before that, a little something to put you on rails. Easy as 1-2-3: hotstteitmegpdnoctnooruglsmloabsnheismralgyeasjlsacshhlwssznlusmablesrafhosuprh

Don't fear the random(). Some even seek for it... VXZDVWWWFYBWYWWCEUWWFXZXZUEYDTETXTXYZDVXVYDYBCYECTXYYDCEFUTUCXVY

Answer 1

I would have expected Vigenère and Playfair ciphers to reduce the skew in the letter distribution, so I think you can rule them out for the time being. Here are some other things you could consider:

1. Perhaps one of the letters corresponds to a space:

   hello world → itssgxvgksr

2. Perhaps there is no letter "e" in the plaintext

   A Void by G. Adair → Q Cgor wn U. Qrqok

3. Perhaps the text was encrypted using a monoalphabetic cipher with homophones (so that two or more letters in the ciphertext correspond to the same letter in the plaintext)

   hello world → itxyg vgkxr

4. Perhaps the text was encrypted with an additional transposition step

   hello world → itssgvgksr → isggstsvkr

5. If the ciphertext contains mixed upper and lower case letters, then perhaps it's just been Base64 encoded:

   hello, world → aGVsbG8sIHdvcmxk

6. If the letters are printed using different styles (e.g., different colours or sizes), then perhaps this is a Baconian cipher

   hello → 00111 00100 01011 01011 01110 → rrvuv ysljq akjsz amshc wvieh

Answer 2

One possibility is that the cipher might be a combination of a transposition cipher and a (probably monoalphabetic) substitution cipher. Since the transposition step will not affect letter frequencies, you cannot tell this combination from a simple substitution cipher based on simple frequency analysis alone.

One way to check whether this might be the case would be to count the frequencies of consecutive pairs of letters and compare this with what you'd expect if the letters were shuffled randomly. (You can do this e.g. with a $\chi^2$ test, or just by plotting the ranked frequencies of letter pairs for the ciphertext and for several randomly shuffled versions of it.) In sufficiently well shuffled ciphertext, the probability of each letter pair should be approximately equal to the product of the individual probabilities of the two letters in the pair; in an unshuffled monoalphabetic substitution ciphertext (just as in plain English text) some letter pairs should be significantly more likely than one would expect based on the individual letter frequencies alone.

Of course, polyalphabetic substitution ciphers will also even out the distribution of letter pairs, just like transposition does, but they'll even out the individual letter frequencies as well. So if the ranked individual letter frequency distribution has a similar shape as for plain English text, but the letter pair frequencies show little or no correlation between successive letters, a transposition + monoalphabetic substitution may be likely.