Say I want to write an encryption algorithm to communicate between me and my friend for this private use. How is that bad?

E.g. I can take the word Hello and encrypt it with a simple algorithm – for example – take each letter and multiply its value by 2 and add 13 so that it becomes completely unreadable.

Is writing a private encryption that stays between two people discouraged or not? What's wrong with it?

  • 4
    Comments are not for extended discussion; this conversation has been moved to chat. – mikeazo Jan 25 '17 at 23:44
  • 4
    Seems this has to be repeated to be clear: Please use the dedicated chatroom for discussions, rants, and/or raves. From this point on, all comments not asking for clarification or suggesting improvements to the question itself can and will be deleted. Thanks for acknowledging… – e-sushi Jan 27 '17 at 14:59
  • 3
    It is not bad to write your own encryption IFF it is for gaining skills in programming/encryption or for fun. What you shouldn't do is to use it for anything sensitive and expect it to be secure. – Viktor Mellgren Feb 2 '17 at 11:31
  • 1
    Request for clarification: do you, or do you not, need the "encryption" to be secure? Is it just a toy between friends for amusement (and for which you don't really care if someone can break the encryption), or do you require it to be unbreakable by various kinds of attacks and attackers? (I get the impression that many of the answers assume the latter; only a few give the obvious answer that there is nothing wrong with a toy understood as such.) – ShreevatsaR Feb 3 '17 at 11:24
  • 1
    As a tangent, you should consider the task of implementing known secure encryption algorithms to be a form of writing your own encryption. It's "easier" to get right, because it's already a solved problem with known correct implementations, but it's just as easy to get wrong, with the added danger that you feel secure because you used AES even though you implemented it in a way that undermines its security. The same caveats apply, all bets are off for educational purposes but don't dare ship something to customers without having it audited. – Jason Feb 3 '17 at 15:15

10 Answers 10

Your question, MikeAzo's comment, and your reply practically could not be a better example of Schneier's Law in practice. Schneier stated:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break.

To answer your reply

How can you break it if I send you this "QTCPIGXKUXTGG" ciphertext encrypted by a merely a simple algorithm which you have no idea about how it was encrypted?

Because even though we might not know exactly what your secret algorithm is, the first thing an attacker is going to reach for are common tools to attack substitution ciphers or polyalphabetic ciphers. Given even a few sentences of ciphertext is likely enough to fully recover every plaintext.

The fact that you don't know how to break it is irrelevant. It's trivial to create a cipher that you yourself can't break, but it's another thing entirely to create a cipher that others can't break. And the odds that you are capable of doing it when you're not aware of even the most basic attacks against ciphers hundreds of years old — not to mention modern concepts like indistinguishability under different attack models — puts you at an insurmountable disadvantage compared to ciphers designed by researchers with decades of experience in the field who are building off of modern notions of security and the discarded remains of thousands of failed ciphers that came before.

As an example, even if your cipher is somehow secure against a ciphertext-only attack (it's not), is it secure if I can trick you into encrypting a message for me? What if I can trick you into decrypting a message for me? What if I know part of or all of one of the messages you send? What if you encrypt multiple messages with the same key?

I'll leave you with another Schneier classic, Memo to the Amateur Cipher Designer:

A cryptographer friend tells the story of an amateur who kept bothering him with the cipher he invented. The cryptographer would break the cipher, the amateur would make a change to "fix" it, and the cryptographer would break it again. This exchange went on a few times until the cryptographer became fed up. When the amateur visited him to hear what the cryptographer thought, the cryptographer put three envelopes face down on the table. "In each of these envelopes is an attack against your cipher. Take one and read it. Don't come back until you've discovered the other two attacks." The amateur was never heard from again.

So here's your first envelope. Given a paragraph or two of ciphertext, your cipher will fail to language-based frequency analysis.

Let me know when you've figured out the other two attacks.

Edit: The comment about indistinguishability under different attack models is one reason why most "decipher this message crypto challenges" are completely bunk. They often simply give an attacker some ciphertext, ask them to decipher it, and declare victory when nobody produces the plaintext after some amount of time. Unfortunately that's not how crypto works in the real world; attackers have many more tricks up their sleeve in practice. They can trick computers into encrypting data of their choosing, they can trick computers into decrypting data of their choosing, and they can usually even do these things thousands, millions, or billions of times. Moxie's post shows how even the most terrible, horribly-designed, and obviously insecure ciphers can be effectively impervious when you restrict an attacker to a single ciphertext-only attack, which aren't representative of attackers' capabilities against ciphers as they're actually deployed in practice.

  • 29
    It's trivial to create a cipher that you yourself can't break: not if you are Bruce Schneier (or Chuck Norris). – mikeazo Jan 25 '17 at 19:05
  • 8
    The way we know a cipher is secure is by observing how it holds up to attacks from the entire cryptography community over years. You can't rely on a cryptographic system unless it has withstood that kind of scrutiny which requires that it be public and that it merit that level of scrutiny. You can develop whatever you want and play with it all you want, but you're a fool if you rely it until it has withstood that level of scrutiny -- it is just way too likely that a motivated opponent can break it. – David Schwartz Jan 25 '17 at 21:11
  • 8
    Only if you assume there's a strict ordering of cryptographers by that metric, which borders on the absurd. You can't strictly order tennis players or chess players on their ability to defeat one another. I think you'd have a hard time demonstrating that of cryptographers too. – Stephen Touset Jan 25 '17 at 23:19
  • 13
    QTCPIGXKUXTGG probably means MARVEL US ALL. – Stig Hemmer Jan 26 '17 at 11:07
  • 11
    @mikeazo: I heard that Bruce Schneier once created a cipher he can't break. And then he broke it. – gexicide Jan 29 '17 at 21:32

I actually think this is a really good question. The answer is because cryptography is a skill, and like any skill, it takes time to develop. Additionally, you will be pitting your (in)experience in the skill against the skills of those who would seek to break your algorithm.

This is the real reason why: It's not that you just shouldn't do it, period. It's that if you do write your own algorithms, you need to realize that it will take a long time before you create anything that is actually capable of securing your information against a dedicated adversary in the real world.

Cryptography is like sword fighting. You would not sharpen a bamboo stick, swing it around a few times, and then go challenge a pack of thugs to a fight. The reason why should be really obvious. Especially when the pack of thugs in question could be a 3-Letter entity such as DJB or other Nation State level Adversaries.

I say this as someone who is written too many crypto algorithms to count on both hands - I have written plenty of algorithms, but proposed and used none of them. Until you have something that 1. Is faster then AES or Salsa/ChaCha 2. Provably more secure then AES or Salsa/ChaCha, why should you propose or use the algorithm in question?

Lastly, you will likely enjoy studying information theory. You will come to understand that keeping the algorithm secret is not efficient because the algorithm itself has a minimum number of bits required to represent it. These bits (the algorithm) simply become the key. It is well established that it is simply better to concentrate your secrecy into a proper small key with a public algorithm. If your key becomes compromised, you simply change the key. If the algorithm is the key, well, you'd need a whole new algorithm.

tl;dr

I disagree that you should not write your own algorithms, it's that you need to have a good explicit reason for using/proposing your own algorithms. I personally would encourage you to write your own, as it will teach you to understand what does and what does not work and why. Doing so will help you to understand various sorts of mathematics and information theory (basically all of the math that I know I learned because/for cryptography).

  • 38
    +1 for pointing out the important distinction between building your own algorithms (and breaking them!) and actually using them in practice. Build away to your heart's content, but don't be naïve enough to use it for anything meaningful, and please don't be the kind of wildly irresponsible person that publishes it alongside with a fancy-looking webpage that encourages other people to use it. – Stephen Touset Jan 25 '17 at 21:24
  • 4
    And another +1 for mentioning the Kolmogorov Complexity explanation of Kerckhoffs' Principle. Great point. – outis nihil Jan 25 '17 at 21:57
  • 19
    An excellent real-life example of making up an encryption algorithm is Solitaire, an encryption algortihm developed by Bruce Schneier for Neil Stephenson's book Cryptonomicon. He, being an expert in the field, designed it to be more secure than the average pen and paper cypher, but explicitly made no guarantees. It took several years of the public poking at it before someone discovered issues with the symmetry of the Jokers which leaked a bit of key data every now and then. Even Bruce himself had no reason to assume the algorithm was flawed until long after the book was in print. – Cort Ammon Jan 25 '17 at 22:04
  • 5
    I'm not entirely sure what that has to do with anything I said in my comment. That said, highly-optimized versions of ChaCha20 are four cycles per byte or even faster on modern hardware. You can even use reduced-round versions (ChaCha12 or ChaCha8) that squeeze in at under 2 cycles per byte. There's no excuse at this point for rolling your own crypto to use in a production device. – Stephen Touset Jan 25 '17 at 23:15
  • 3
    Equivalently: there's no reason not to construct your own parachutes as a learning exercise, but there are good reasons not to jump out of planes wearing them. Stick to throwing eggs off the balcony. There's an additional difficulty with crypto, that you can't test whether your algorithm is good or bad anything like as easily as you can throw an egg off a balcony and observe whether or not it smashes. – Steve Jessop Jan 31 '17 at 11:26

It's discouraged because (A) it's statistically unlikely that a beginner comes up with a strong cipher and (B) lacking a working knowledge of cryptanalytic techniques, a beginner won't know that their cipher is vulnerable.

With most engineering problems outside of security, it will be obvious when things aren't working. If your video codec doesn't work you'll have strange green lines all over the display; if your stairs sensor doesn't work, the robot falls down the stairs, etc. But if your cipher is cryptographically vulnerable, you will still be able to encrypt and decrypt messages just fine. The only problem will be that people without the key can also decrypt the messages, and they won't necessarily be eager to inform you of this.

  • 1
    Extremely good point. Over on StackOverflow about 90% of cryptography "works". Less than 10% is actually secure though, which is kind-of-a bummer. I've pointed out vulnerabilities in answers with more than 90 upvotes, and no downvotes because they worked. – Maarten Bodewes Feb 1 '17 at 16:13

For the same reason that writing your own compiler, framework, or operating system is "discouraged". If you want to do it as a learning experience, for personal satisfaction, or curiosity, go for it. If you want something that actually serves the purpose for which it is intended and you want it sooner than years from now, understand that there are already people who have spend long careers on this and have since made the many, many thousands of mistakes that you will make in the course of producing something much less solid than what they already have.

I would not "discourage" you from writing your own symphony, but if you have not studied music theory, composition and instrumentation, I'm probably not going to be interested in sitting through it, and I doubt if anyone other than your doting mother would be either..

  • 7
    +1! Linus Torvalds kind-of did that, he wrote an operating system just out of curiosity. BUT! He is the exception to the rule, AND he accidentally stumbled into a big void in the market, AND he was quickly joined by an ever-growing team, AND it still took 10-20 years to get production-ready (depending on your exact definition of "production" and "ready"). – Jörg W Mittag Jan 27 '17 at 0:59
  • 11
    The big difference is that you notice when you fail at one of the above problems. Bad encryption on the other hand works fine but just isn't secure. – CodesInChaos Jan 27 '17 at 9:56
  • 2
    I disagree with this analogy. Any competent programmer can write an invertible function from strings to strings in half an hour, and think they have a secure cypher. The amount of effort required to get a compiler or operating system to the point where you might even think about security vulnerabilities is many orders of magnitude greater. – David Richerby Jan 27 '17 at 13:32
  • 7
    @JörgWMittag Linus Torvalds actually just wrote the kernel. And he wasn't joined by an ever-growing team; that team was already there and had created hundreds of programs for the GNU operating system. They didn't have a kernel though, so when Linus bundled the operating system with Linux as a demonstration of the kernel, people thought the whole thing was called Linux and that it was all created by Linus Torvalds, which the people who worked on the GNU project are annoyed about (because Linus' ideals are different from theirs, which were the reason they created GNU). Some call it GNU/Linux. – wizzwizz4 Jan 28 '17 at 8:16
  • 4
    It's actually GNU/Mozilla/Google/Apache/Oracle/.../Linux, thank you very much. /s – immibis Jan 30 '17 at 5:23

In addition to the great answers on cipher complexity, I think it should be mentioned that the security engineering of the code that implements and utilizes ciphers - things like key exchange, key selection, and the generation of entropy - is very hard as well. Ross Anderson has a terrific book on Security Engineering, but an earlier essay of his (from 1993!) introduces some of the main points specific to cryptography (using old ATM security models as his case study), "Why Cryptosystems Fail" (1st Conf.- Computer and Comm. Security ’93

  • It may be very hard, but remember that it's just programming, and no harder than most other professional coding projects. How many people here could add another weapon to Halo 5? Similarly, who here could code a proper elliptical crypto application? There is a huuuge tendency herein to deify cryptography above what it actually is. Also, the failure of a crypto app is mostly fairly limited in comparison to failures in the real world. And how else do you gain experience? – Paul Uszak Jan 31 '17 at 5:05
  • 2
    @PaulUszak I'm sure most professional programmers could, given sufficient time/motivation, implement an elliptical crypto application that would be able to encrypt and decrypt data. However, such an application would inevitably be vulnerable to a number of side-channel attacks, and as such would not actually be secure. Implementing crypto isn't inherently that difficult, but securely implementing crypto is, and so is discouraged for the same reason as inventing your own cryptosystem. – g.rocket Jan 31 '17 at 18:25
  • @g.rocket Are you now proposing a ban on implementing crypto because it will be insecure? That would be a new development even for this forum. – Paul Uszak Jan 31 '17 at 19:26
  • 2
    @PaulUszak If your implementing crypto to learn about it, go ahead. But in a production application, if at all possible, use a quality crypto library that's been written and verified by security experts. There's a lot of tricky problems besides the actual cypher that can make your crypto insecure. stackoverflow.com/questions/1914257/… – g.rocket Jan 31 '17 at 21:04
  • 3
    @PaulUszak The primary difference with information security (crypto or otherwise) is that a failed implementation usually works just as well as a successful one; it's not obvious that what you've done hasn't worked until someone has exploited it and it's too late. – Xiong Chiamiov Jan 31 '17 at 21:18

It is discouraged, for all the good reasons the people already answered you (so I won't repeat).

There is a twist, however (which is the point of this answer) - no matter how weak your cipher is (and it IS weak), it has two advantages going for it:

  • it is more secure than plaintext that you had before (even if only by slight margin, as you're not experienced in crypto).

and (this one is going to be controversial)

  • it might actually give you more of a chance of not being spied upon if you're not already interesting target for NSA or some other agency. (Now before you people downvote me to hell with "security though obscurity doesn't work" please allow me to explain :)

As mentioned above, if you're specifically targeted, using toy-cipher won't help you at all and you're practically as unprotected as when you were using unencrypted communications (or even slightly worse, as you might take more risk talking when believing you're protected).

However, beside (or one might argue: before) security of the cipher itself, and security of its implementation (those are two very different things, remember Debian GNU/Linux OpenSSL fiasco for example) there is one other key factor in determining if your communication is going to be de-privatized. And that is, "will anyone try"? And someone will try if any of the 3 things are true:

  1. you're interesting target yourself (for example: top politician, financial access / banks, celebrity status, have access to rare information, etc)
  2. you're perhaps not interesting yourself, but there is a lot of "you" using same security method. In other words: popularity. Sure, facebook does not contain much of top secret data, but having a way to access ANYONEs facebook account is going to be very good incentive for hackers to try to crack its protections .
  3. bad luck (eg. some hacker is bored and just happen to take a pick on your site, transforming it to point 1. above)

Now, sure TLS1.3+AES256-GCM/SHA512 is great thing today. Nobody can spy on you (except NSA and friends, but if you made enemy of them, you're done already). But there is enormous amount of people using it, and thus it becomes enormously profitable to crack it. So it will happen.

Not today, not in 5 years perhaps, but in 50 years random script kiddie would probably be able to decrypt your communications (as you didn't use PFS, doh!) with a simple click.

However, your toy-crypto chat, used in the whole world only by you and your friend to talk about slightly embarrassing fetishes, will likely remain hidden from everyone but legitimate parties for all eternity. Simply because nobody cared enough to give it minimum of effort needed to crack it, and it wasn't popular enough that is was decrypted by default by some PRISM-like mass surveillance.

But the other answers had it correct; best practice for continued secret communications currently is: "a) don't make enemies of various TLA government agencies. b) use secure popular ciphers implemented by competent programmers. c) keep upgrading both ciphers and implementations until at least the day you die" - although it is usually (with perilous results) often shortened just to half of point (b) "use secure popular ciphers".

But I'm wondering if in the long run we wouldn't all be much better if we had billions of obscure and insecure ciphers each used only by few people; as apposed to few supposedly uncrackable ones that everybody uses (but which are secretly flawed - which is known only by 1984-ish goverments and other murky elements.

As (anecdotal) "evidence", every month I help fix dozens of broken wordpress/joomla/etc sites which were "sooo secure" just a few months/years ago. But strangely enough, every now and then I stumble upon properly written (eg. verifies input :) old perl or even shell scripts, using insecure RC2 or even just hackish XOR, which still stand strong after decades. Just because there were different (and not popular) and nobody cared enough to spend time to crack them.

As your biggest problem nowdays is probably not that you will get targeted, but instead that some automated bot will exploit some hole in popular piece of software which you didn't patch quick enough. As there is limited amount of crypto-crackers, and there are most of the time chasing bigger cat than you. So you'd most likely become low-priority ticket and thus fade into oblivion.

  • 6
    "it is more secure than plaintext that you had before" - This is disputable. You might be tempted to discuss more private things over your seemingly "safe" channel (a theory known as risk compensation). – Dubu Jan 26 '17 at 10:04
  • 3
    "it might actually give you more of a chance of not being spied upon" - The opposite might also be true. As the NSA rules apparently say: maintenance of technical data bases requires retention of all communications that are enciphered. So you might evade an XKeyscore analysis of your data, but it will probably be stored (and analyzed) because it is encrypted. – Dubu Jan 26 '17 at 10:12
  • @dabu Didn't I already addressed risk compensation with "[...] using toy-cipher won't help you at all and you're practically as unprotected as when you were using unencrypted communications (or even slightly worse, as you might take more risk talking when believing you're protected)" ? – Matija Nalis Jan 26 '17 at 13:38
  • 1
    Nitpick: TLS 1.3 requires PFS. – Maarten Bodewes Feb 1 '17 at 16:19
  • 1
    @MaartenBodewes true, nice catch. Although there is still 0-RTT resumption which is still non-FS, right? – Matija Nalis Feb 1 '17 at 19:56

Obscurity can offer some security, but its usefulness is very limited. Among other things:

  1. It is difficulty to quantify how much security it offers. Since human beings make very bad random number generators, it's hard for someone who comes up with a seemingly-original idea to know how likely it is that someone else would think of it.

  2. Although it is not difficult to come up with an algorithm which will be secure under the right conditions, maintaining those conditions is difficult. The algorithm and all plaintexts must be forever kept secret from anyone who wouldn't be trusted with the content of all messages that have ever been sent using it. Further, the total volume of ciphertext available must be kept fairly small, but it's difficult to quantify how small.

  3. The only way to compartmentalize risk is to use cryptographically-independent algorithms for every different kind of message whose disclosure shouldn't affect any others. Devising truly independent algorithms is difficult for reason #1 above, and most approaches that rely upon security through obscurity fail to properly compartmentalize risk as a result.

Although things like DES, AES, etc. are often described as "algorithms", it would perhaps be more useful to think of them as "algorithm generators". If one has a good source of 256-bit random numbers that have essentially zero probability of matching anything else anywhere in the universe, then "use AES with this number I just generated" can be used as an encryption algorithm which is cryptographically independent from any other algorithm that has ever existed. Consequently, it's much easier to compartmentalize risk using such approaches than by devising one's own encryption schemes.

While I agree with the answers already provided, I'd like to add two more points.

The first one is a quote from Bruce Schneier (yeah, him again):

There are two types of encryption: one that will prevent your sister from reading your diary and one that will prevent your government.

So, if confidentiality is actually not a big deal and your only concern is that your texts will not be referenced by Google, you may use your own home-made cryptosystem. I read that some adult newsgroups used ROT13 exactly for this reason: no confidentiality was added, but no one (especially children) would stumble upon the content by chance.

However, if you do so you must assume that someone has broken your system from day 1 and that anyone who wants to break it, actually can. It's all the more true that a cyphertext is similar to a huge billboard reading: "Decrypt me if you can".

If you want real confidentiality, however, then don't use a cryptosystem which has not been tested and approved by experts in the field. See other answers if you're not convinced why.


The second point is related to your comment: "How can you break it if I send you this "QTCPIGXKUXTGG" ciphertext encrypted by a merely a simple algorithm which you have no idea about how it was encrypted?"

This is not compliant with Kerckhoff's principle:

[A cryptosystem's algorithm] should not require secrecy, and it should not be a problem if it falls into enemy hands

Or, more clearly, if your security relies on the fact that your enemy doesn't know the algorithm you used, then your security is crap. The attacker is perfectly able to find a fatal flaw even without knowing your algorithm.

The obvious answer is that nothing is wrong with making a toy cipher to exchange messages with a friend -- just like nothing is wrong with using gets() in a hangman game that you and a friend are going to play. Bruce Schneier isn't going to break down your door and confiscate your computer. What you do with your friends is your own business.

Important qualifications:

  1. It would be extremely unwise to think of something like multiplying the value of a letter by 2 and adding 13 as anything more than a toy.

  2. What would actually be wrong to the point of professional malpractice would be if you use ad-hoc encryption on a product that you foist on unsuspecting clients (just like it would be wrong if you use gets() in any production code).

  3. While there is nothing wrong per se with playing around with toy ciphers or writing private code that is vulnerable to buffer-overflows, doing such things can easily become a habit and habits are hard to break. If you get use to using gets() (a notorious source of security holes but easier to use than secure alternatives) for your own toy projects then you will be tempted to cut corners and use it in production code. Similarly, if you get in the habit of thinking that it is easy to disguise messages then you might be tempted to write your own crypto in production code rather than doing the tedious work of learning how to properly use an industrial-strength crypto API.

To sum up -- nothing is wrong per se with the scenario that you describe (toy crypto between friends) but it nevertheless raises warning flags since it can drift into something which is wrong.


gets API (removed from the latest C specifications and deprecated in C++):

Reads characters from the standard input (stdin) and stores them as a C string into str until a newline character or the end-of-file is reached ... and does not allow to specify a maximum size for str (which can lead to buffer overflows)

Using gets is almost guaranteed to make the system vulnerable to attacks.

Crypto is hard, and very easy to get fatally wrong. The chance to win the lottery ten times in a row when you're not even buying tickets is higher then the chance that you will come up with even a halfway passable encryption.

But since crypto is hard, you won't notice that your encryption will take any hobby cryptographer all of half an hour to break. This is not just because algorithms are hard to come up with, there are also modes, key lengths, key exchange and a hundred other things to consider for a full encryption scheme. And even if your scheme is fine on paper, you still need to make an implementation that doesn't add flaws (such as weak IV generation, for example).

Honestly, asking why making your own crypto is discouraged is similar to asking why letting the passengers fly that Airbus 380 is a bad idea. If you actually are a cryptographer, you can play with invention new encryption schemes. If you are not, sit back, enjoy the in-flight entertainment, and let the people who have a couple thousand hours of experience handle the tricky part.

protected by e-sushi Jan 27 '17 at 14:50

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.