# Breaking Double Encryption

I am trying to understand how an attacker knows when he has successfully decrypted a ciphertext for an assignment. As such, some pointers/hints for the following questions would be greatly appreciated.

1. If you double encrypt secret data with two different algorithms. How does the attacker know if he already broke the 'first layer' of encryption?

2. Is it possible to determine the encryption algorithm used by analyzing the cipher text?

3. If you find out what algorithm was used does it give you the necessary information to know when you successfully broke the 'first layer'?

4. Considering the above, what difference would it make if the attacker knows the open-source program or the (public available) algorithms used to protect the data?

To begin with 4: Remember Kerckhoff's principle. You should always assume that the attacker knows which algorithm is used to encrypt your data. All the algorithms used in practice are designed to be secure under this assumption, so you should consider that hiding the algorithm from the attacker is superfluous. But as a hypothetical...

1. I can't think of any real-world cryptosystem where it would be possible to tell exactly which algorithm was used knowing only the ciphertext, but in a theoretical sense such an algorithm could exist, and would not be any less secure because the attacker is supposed to know that anyway.

2. Same as above, for mostly the same reasons.

3. In light of the above, it shouldn't make a significant difference. If it does, your algorithm relies on security by obscurity, and is generally worthless.

In your "double encryption" case, the attacker will just treat the composition of the two algorithms as one big algorithm, he will not try to decipher one and then the other.

• "In your "double encryption" case, the attacker will just treat the composition of the two algorithms as one big algorithm, he will not try to decipher one and then the other." This is quite possibly untrue: assuming the two algorithms use at least partially different keys, an attacker would perform a Meet in the middle (MITM) attack – figlesquidge Mar 10 '14 at 18:16
• To 1, I believe RC4 has enough output biases that it should be possible to detect its use with a small number of ciphertexts. – Stephen Touset Sep 8 '14 at 20:43

In general (especially without knowledge what encryption you consider), it's not possible to detect "correct decryption of one layer", if that's all you have and this "middle ciphertext" is not in a specific format.

However, from today's point of view this is almost entirely irrelevant, because stronger attacks are considered:

• Kerckhoff's principle states that security should never be based on not knowing the algorithm. That is basically the weakness of most historical ciphers. Security must stem from the key alone.
• You seem to consider "ciphertext only" attacks, which is just too weak for today's standards. For symmetric encryption you can also consider known plaintext attacks (attacker gets pairs of ciphertext and plaintext) or chosen plaintext attacks (where the attacker chooses what plaintext is encrypted and gets the ciphertext). The later is actually needed to be called "secure" today.
• As a rule of thumb: Unless you clearly know what you are doing, just combining two encryption schemes does not necessarily make it more secure. There is no general statement possible, but you might add a security weakness unintentionally.
• >> but you might add a security weakness unintentionally. <Sudhakar>I totally agree with you on this point.</Sudhakar> – Sudhakar Chavali Jul 11 '16 at 17:59