About Cryptography in a Character Language

Question

Suppose I had a message in Chinese (or another non-phonetic language) and I wanted to encipher it.

Some of the simplest encryptions in English are substitution ciphers, but such ciphers don't seem to be a viable option for a language such as chinese (it's more similar to a code...). What would be a simple cipher for such an alphabet? (Lets assume that we don't want to transliterate it to English first).

Second, suppose that someone had used a substitution cipher on such a language (in the code sense where every word stood for another).

• Would frequency analysis still work?
• Are non-phonetic languages more or less secure when used with substitution ciphers?
• Assuming frequency analysis is not an option, how would one break it?

Answer 1

For designing a cipher, one first has to decide about the alphabet.

This is a bit problematic for a language like Chinese, since it is not really clear how many (and which) characters should be used. The number of signs known by people differs greatly. You don't want that your encrypted message is un-decryptable just because you used some unknown character in the ciphertext. One possibility here would be to restrict the ciphertext alphabet to some "known to everyone" subset (while still allowing a greater alphabet on the plaintext side).

Assuming we have decided on an alphabet, we also should decide on some encoding of this alphabet, i.e. some ordering of the signs, or a mapping to numbers. This allows using some simple mathematical formula/algorithm for the actual encryption, instead of having to use a "translation table" as the key (which would be quite big in the case of a big alphabet).

Every monoalphabetic substitution cipher can be presented as a "big table", and thus breaking the big table algorithm also breaks each other (monoalphabetic) algorithm (which in fact are only ways to write the big table in a shorter way).

So, how to break a monoalphabetic cipher? Just like always. In a language like Chinese there are more frequent words (= signs), too, and more or less frequent two-word or three-word combinations, just like in our European single-sound-alphabets, too. The problem is just that there are more different signs, and thus you need a lot more text to get significant samples. Then we try the most probable words first, and try to fill in the rest, and see if it starts to make sense. Some signs used only once or twice are quite probably not decodable at all, if multiple different ones would make sense here.

The reason that modern substitution ciphers on really large alphabets (like AES) are comparatively quite secure even in ECB mode (in other modes they are not pure substitution ciphers), is that you don't get as much blocks to find many repeated ones - and that you simply can't write down the whole substitution table, for space and time reasons.

Answer 2

non-phonetic language

I think you want to use the term "non-alphabetic language". Chinese, like Japanese and Sanskrit is a syllabic language, where the tokens refer to syllables. Chinese, unlike most western languages is tonal. There is an example in the "mechanics" part of that wikipedia page that describes the syllable "ma" in Mandarin Chinese, and how it can mean either of 5 words depending on pitch.

There are a number of books that discuss the Japanese codes and ciphers of the WW2 and prior era (for example, The Emperor's Codes, and Codebreakers: The Inside Story of Bletchley Park). In their case they used a number of schemes. Substitution codes (such as JN-25) would be based on words, with multiple substitutions available for each word in an attempt to reduce the possibility of frequency analysis (the words would map to 5-7 digits depending on the particular code). One snippet of a page showing the number to Japanese step is on this page. One whole page of the same step. JN-25 used groups of 5, and each group was divisible by 3 - an error correcting scheme that reduced the possible code space by 2/3, leaving only 33,333 possible code groups. The code groups would then be super-enciphered by adding numbers from another book. The main kana to code group to kana book did not change much, the super-enciphering and "formulas" for standard messages are what changed from time to time.

Other systems, such as Purple, used a phonetic transliteration to Katakana, then using a Roman alphabet keyboard to encode the syllables. You may notice that the syllables consist of one consonant plus one vowel (the exceptions are the vowels by themselves, and the consonant N {which is sometimes M before certain other consonants}). Katakana has more sounds than Hiragana because it is the syllabary devoted to foreign words, so it can represent sounds not native to Japanese (such as V or L). One of the weaknesses of Purple was that the vowels and consonants used different stepping systems.

Converting the native language to something that can use Roman letters was useful in the days of Morse and Telegraph, but a more modern system would most likely use Unicode and encipher those values. One book I remember reading back in the days before Unicode is CJK, and handling Japanese (which I was studying at the time) was a problem before Windows XP.

Answer 3

If you consider your message as a series of Unicode code points (aka characters) then it does not matter that it is really pictograms you are encrypting but "letters" in a very large alphabet. Then you can use substitution or transposition or just even whatever modern cipher that work on bytes or characters.

Answer 4

"What is the simple cipher for such an alphabet?"

I'm assuming you have some message written by paper in a written language with thousands of "characters", and you want to write an encrypted message on paper using more-or-less the same set of characters.

One could use a substitution cipher, as you suggested, using a codebook that, for each red plaintext character, lists some substitute black encrypted character. Such a cipher would be more-or-less equivalent to a nomenclator cipher, which, as you suspected, is significantly more difficult to crack than a Latin-character monosubstitution cipher. A codebook of ten thousand symbols would be no more difficult to use than English-language commercial telegraph codes of ten thousand words -- in both cases you may need to paraphrase the message so that the plaintext only used the available red plaintext characters/words, avoiding the (hopefully rare) words/symbols left out of the codebook.

My understanding is that two Chinese telegraph operators, at opposite ends of a telegraph link, use two identical dictionaries with (nearly) every Chinese character printed with its corresponding 4 decimal digit Chinese telegraph code. Once a message in Chinese characters has been translated into a series of decimal digits, one can use any of the classic ciphers on them ... and perhaps translate the resulting encrypted decimal digits, 4 at a time, back into Chinese characters.

My understanding is that Japanese telegraph operators use a completely different system: they write down the message entirely phonetically, using only katakana letters, of which there are around 40. That's few enough letters that any of the classic ciphers can be used with them directly.

However, other ciphers may be much easier to use, because they avoid the necessity of a codebook, and support using any conceivable Chinese, Japanese, or other character:

A null cipher mixes the characters of the message with many null characters (chaff) that are ignored when deciphering.

A transposition cipher, such as a grille cipher or the "Chinese cipher", shuffles the characters around. While a transposition cipher alone may be inadequate for a short message in Chinese-like characters, mixing in some null characters might give adequate security against manual decryption.