13
$\begingroup$

I asked a question on security.stackexchange, but was told it would be a better fit here:

https://security.stackexchange.com/questions/32779/how-would-one-crack-a-weak-but-unknown-encryption-protocol

My question is: Assume I'm an attacker trying to break a weak-but-unknown encryption algorithm. i.e., someone not too bright made a home-brew encryption algorithm, and I'm trying to break it.

BIG FAT DISCLAIMER: Using home-brew security algorithms is very bad for many reasons. Everybody in the security community agrees about it. It's very hard to keep algorithms secret. Please don't answer this question explaining why it's a bad idea to use a home-brew algorithm. I know it's bad. Assume the algorithm is completely unknown.

Okay, now back to my question.

How would you approach a completely mysterious ciphertext when you have no idea what algorithm was used to make it? For me it would look like a meaningless stream of hex, but I'm very curious to know if there's a method to analyze such data and try to break the code.

Here's an example of such a ciphertext.

Is there a way to break it using a single ciphertext? If that's very difficult, I'll be happy to learn how to do it from multiple ciphertexts. I can create more ciphertexts using the same home-brew algorithm, if someone here thinks he can use that to figure out the algorithm.

|improve this question
$\endgroup$
  • 1
    $\begingroup$ It's not so much direct analysis on the ciphertext, which doesn't work unless the encryption algorithm is truly crap (for instance, there is no easy distinguisher between, say, AES and Blowfish). The real danger is someone disassembling software which uses your "unknown" protocol and figures out how it works, at which point you better have a secure key to fall back on, or you're screwed. A single ciphertext won't tell you much if your target's cryptographic skills are beyond ROT13 level. $\endgroup$ – Thomas Mar 20 '13 at 4:52
  • $\begingroup$ What does "weak" mean? In theory if it's mean that there exists a poly-time algorithm that "invert" the encryption then in theory you could "learn" the function, if you have an oracle access to it. In practice i would try different a machine learning algorithm and see what happens! $\endgroup$ – AntonioFa Mar 20 '13 at 18:16
  • $\begingroup$ Please don't post the same question on two sites. Instead, flag one of them to ask the moderator to move it to the other site (you can click the "flag" button to ask that it be moved). We don't want a copy of the same question on two different sites. $\endgroup$ – D.W. Mar 21 '13 at 0:00
  • $\begingroup$ Do you have a bunch of ciphertexts? $\endgroup$ – Fred Concklin Mar 18 '15 at 14:09
9
$\begingroup$

It depends on what information you have. If all you have is that one ciphertext and no way of acquiring more, there's not much you can do.

If you have multiple ciphertexts, you can try to observe any patterns in them. If you have the corresponding plaintexts, you can really start to make some useful observations. If you have some plaintext, ciphertext pairs, you can try to find patterns in the data.

An even stronger attack would having access to the encryption oracle. That is, being able to encrypt plaintexts of your choosing. Here you could try special encryption strings to see if they leak any information about the algorithm. You can try things like "a", "aaaaaaa", or string of null bytes and see what information you can extract about the algorithm.

If the encryption algorithm is insecure, your goal should be to figure out what the algorithm does. Once you know how the algorithm works, you can presumably figure out a way to break ciphertexts or get the key.

|improve this answer
$\endgroup$
  • 4
    $\begingroup$ This is it. From even a couple of ciphertexts you wont get much. almost anything is secure against ciphertext only attacks. The next step is knowledge about the plaintext (distribution, common patterns, etc), and then the known plaintext attack. At this point most historical algorithms cracked (Enigma etc.). $\endgroup$ – tylo Mar 22 '13 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.