One Time Pads and reuse of the decrypted message?

How is it insecure when Alice encrypts a message with a One-Time-Pad to Bob, and Bob then uses the decrypted message from Alice as the next One-Time-Pad?

For example: Alice sends Bob a message Hello World with the Key of Orange Fish which they both shared earlier. Vvlyu Atzdk (Hello World). Then Bob uses Hello Word (which he decrypted with Orange Fish) as the key for the next message.

If Bob’s message back to Alice Hello Alice encrypted with Hello World to Oiwwc Wzznh (Hello Alice). I’m sure this is detectable, but I’m curious how it could be broken.

• That would probably end up using a sensible message as the pad. $\;$ – user991 Oct 24 '15 at 4:42

First things first: what you are describing there is not a one-time-pad!

Per definition, OTP requires the “key“ to be…

1. a truly random one-time pad value,
2. generated and exchanged in a secure way,
3. at least as long as the message, and
4. only to be used once.

What you describe (eg: using a key going Orange Fish) can merely be compared to a simple, keyed substitution cipher with security boundries comparable to a regular Ceasar cipher. Differently worded: your encryption technique is no where near the cryptographic security a real one-time-pad can provide.

Now, to the core of your question: if Bob would base his key on the plaintext of what Alice send him, you would practically be creating a ciphertext by combining two plaintexts (which should actually remain secret).

To attack such a simple “cipher“, an attacker could simply exploit the statistical bias coming from the fact that both the key as well as the ciphertext are regular, human language. Letter frequencies et al would quickly help differring the key from the plaintext. The fact that the space character doesn’t even get encrypted, makes it even easier to recover the plaintexts. The result of using such an “encryption“ would be that an opponent would quickly be able to successfully attack it, and recover both the message Alice tried to secretly send to Bob, as well as the message Bob secretly tried to send to Alice as a reply.

Simpler said: your idea incorporates more than the usual danger of leaking a single secret… because it practically risks the leaking of two secrets: the plaintext which is being send, and the previously received plaintext which is reused abused as key to encrypt the plaintext to be send.

As for an example of the cryptanalytic side of things, I’ll refer you to this question and its accepted answer, as they provide a lengthly and indeep insight on how one could extract a keyword from a keyword cipher – including a practical example. When reading that Q&A, keep in mind that your “algorithm” may seem similar, but actually is much simpler and incorporates more insecurities than what they are talking about there.

Long story short: the fact that Bob and Alice are risking to leak both their secrets twice is pretty inacceptable from a cryptographic point of view. Yet, I am sure every potential attacker will be more than happy when you allow him to gain a complete transcript of your communication in such an simple way; especially since you are including all those nice carbon-copies of the plaintext, which are (ab)used as keys.

Honestly, if I were a cryptanalytic opponent facing your idea in a practical situation, I’ld probably put you on my Christmas-card list for providing me with one of the easiest and most amusing jobs ever.

• Thank you esushi for the detailed reply. Very helpful and it made a lot of sense. Sorry for not accepting it sooner my email wasn't showing replys. – cryptodragon Nov 5 '15 at 5:51
• @cryptodragon No need to be sorry for anything… I’m glad my answer was helful enough to be acceptable. So, thank you! Oh, by the way: Welcome to Crypto.SE! ;) – e-sushi Nov 5 '15 at 5:54

This would be insecure in most any practical setting. This is because most any message has patterns or structure that is known or guessable to the attacker. For example, perhaps many messages start with "Hello" and end with the sender's name ("Sincerely, Bob"). With this knowledge, an attacker can not only determine that part of the key to the next message, but also that part of the plaintext in the next message, and therefore that part of the key to the message following that... etc. Essentially, if the attacker can guess any part of the message, the attacker can break that part of all future messages.

Even worse, the attacker can do the same thing going backwards. Given an encrypted message ("Oiwwc Wzznh") and part of the corresponding message ("Hello"), the attacker can figure out that part of plaintext from the last message as well! Similarly, the attacker can break that part of all previous messages as well.

In modern cryptography, this is called a "known plaintext attack". Cryptographic algorithms like AES are carefully designed to avoid these kinds of attacks.

Consider why a One-Time Pad has the words "one time" in the name. An attacker with ciphertexts from a two-time pad can calculate:

$$Plaintext_1 \oplus KEY \oplus Plaintext_2 \oplus KEY$$

which equals

$$Plaintext_1 \oplus Plaintext_2$$

which can be broken via various statistical (or simple guesswork) methods.

With this scheme, the attacker has access to $Plaintext_1 \oplus Plaintext_2$ directly!