After reading up on the one-time pad (OTP) encryption method, I could see how it would offer unbreakable encryption if used properly.

Moreover, I looked at how the OTP could be broken if the OTP-key is used on more than one message.

What I now wonder is this: if classically you use a Vigenère Square to substitute letters according to your OTP-key, you can break the encryption by finding cribs and then applying the substitution of the letter in the second message. This is possible since I know the substitution scheme.

Now the interesting point: if I randomize my substitution table, I would think that a decryption using the same OTP-key would no longer be possible, unless I have enough messages to successfully map (almost) each letter to this unknown table.



1 Answer 1


Bad idea (as far as I know & understood).

First, let's make sure you do not confuse Vernam cipher (XOR) with OTP.

  1. OTP (One Time Pad) requires you – as described here – to use (1) a truly random one-time pad value, (2) generated and exchanged in a secure way, (3) at least as long as the message, and (4) only to be used once. So, what you are describing is not a One-Time-Pad.

  2. Vernam cipher applies this concept with a stream cipher (simple XOR)

The best example of “miss-using” a OTP is usually given as:

$c_1 = m_1 \oplus k\\ c_2 = m_2 \oplus k\\ c_1 \oplus c_2 = m_1 \oplus k \oplus m_2 \oplus k = m_1 \oplus m_2$.

where using the same key twice defeats its purpose and provides interesting information on the plain text (statistical attack).

Why your idea is bad ?

Given Kerckhoffs' principle:

  • The system must be practically, if not mathematically, indecipherable;
  • It should not require secrecy, and it should not be a problem if it falls into enemy hands;
  • It must be possible to communicate and remember the key without using written notes, and correspondents must be able to change or modify it at will;
  • It must be applicable to telegraph communications;
  • It must be portable, and should not require several persons to handle or operate;
  • Lastly, given the circumstances in which it is to be used, the system must be easy to use and should not be stressful to use or require its users to know and comply with a long list of rules.

Therefore you must consider that your Vigenere substitution table has to be publicly known : The secret should only rely on the OTP value. Hence the point of randomizing your substitution table is not that interesting any more.

Therefore, given the known substitution, statistical properties can be found that would defeat the randomization idea.

By the way, this idea is close to the design of Enigma, where the substitution table changes every round defined by an initial key, which is kept over the course of a day. Have a guess what happened during WW2... What would have happened if the Germans had changed their keys between every transmission?

PS: Rolling your own crypto is always a bad idea.

  • $\begingroup$ Thanks for the detailed answer! I misunderstood something in how the XOR-ing works. Also a good point in keeping the substitution table public. Is there an example somewhere out there where the XOR'ing is a bit simplified? I only found examples using HEX which is a bit hard to get into. $\endgroup$
    – Horst
    Feb 19, 2016 at 18:17
  • $\begingroup$ @Horst: en.wikibooks.org/wiki/Cryptography/One_time_pads never uses HEX in any of its examples, and has a bunch of links that may be easier to understand. $\endgroup$
    – David Cary
    Jan 16, 2023 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.