Everyone says you should never build your own cryptosystem. That doing so is hard and the only way you know it's secure is if it stands up to serious scrutiny. And yet, if you are willing to go way overboard on key size, rounds cpu time, memory etc. Would it not be fairly easy for even a novice cryptographer to build something secure. Especially if using a well understood basic construction(e.g feistal network).
If you are building something based on existing research, then yes, the construction of such a system would not be challenging. As a very basic example, it wouldn't be extremely challenging to implement a system based on the One-Time Pad given that you have a good programming background and assuming you follow the algorithm's stipulations correctly (e.g. avoiding repeated keys for new messages, good RNG, etc.), your system will be provably secure.
However, if you seek to design and develop new encryption schemes, you might have a harder time. You will likely encounter issues in multiple layers starting from the theoretical development and proof of security and all the way to the practical implementation of the scheme. That is, you will have to, firstly, design an encryption algorithm that can be proved to be secure in a mathematical way. Secondly, depending on the complexity of the algorithm that you designed, you will have to be extra careful in avoiding bugs in your code that could allow a motivated attacker to break your crypto system.
Because of these reasons, it is highly encouraged that such algorithms are open: it allows for more eyes to look for both flaws in the theory as well as in practice.
Some resources to expand your research:
The widely-accepted ciphers like AES represent an optimal balancing of multiple factors. They provide the required level of security while providing good performance. There are quality implementations that avoid side-channel leakages. They are trusted enough that protocol designers will include them.
If you are building your own, you have to start by accepting that you are not going to occupy this same optimal position. Since the most fundamental property of the cipher is providing security, you'll sacrifice along other dimensions: you'll have a much larger key and you'll have more rounds and hope that this can make up for potentially lower security per round. A Feistal structure has 2 main components that you would need to design: the round function and the key schedule. If you are aware of the requirements for a round function, I think you should be able to come up with something not terrible, and a large number of rounds will largely balance any flaws. The key schedule is critical though, since you are going to have a large number of rounds. This is harder to get right.
An amateur who is familiar with modern design philosophies has a good chance. A novice probably couldn't, and also has the problem of over-estimating their ability and not compensating for potential weaknesses.
It seems the answer of this question depends greatly on how you define several key terms. "Secure", "your own" And "Novice cryptographer".
First we need to define secure, Aleph points out that increasing key size won't help because any attack faster then brute force would make the cipher "insecure" from a theoretical perspective. In fact by being creative and throwing in several ciphers one on top of the other though you may increase attack time you are adding more places where you can be insecure. Just doing more may increase attack time but from a theoretical persepective is likely to create a weakness.
The next term is "your own", the closer you stick to well established systems the less likely you are to go wrong. If you read up on how someb magic number of chosen a compatible different one you are not likely to go very wrong, especially if you compensate for any mistakes by increasing rounds and applying the cipher repeatedly with different keys. If you use a basic construct like Feistel neteorks but are very creative in your round function there are more places to go wrong and if you whip up something truely novell you are essentially guranteed to be making huge mistakes.
Lastly there is the issue of novice, someone which took a few college level cryptography courses maybe a graduate level course seems like a good persona to discuss. Such a person though not normally qualifies to particpate in new cipher design will understand the constructs the ideas of defusion, non linearity etc. Someone less qualified is obviously more likely to make a silly mistake.
Lastly we need to remember even very smart people can make silly mistakes and without proper review of both algorithm and implementation you can never be sure.
So I would answer yes, someone with basic college level cryptography training, not straying too far from well studied ciphers and taking plenty of security margins is likely to be able to produce something practically secure.
Don't try this for anything important.
Yes, in the same way as one could study a medical textbook and perform their own surgery. It's not that scraping out a tumor is rocket science, it's that there's a lot of bad things that can happen before, during, and after the easy part. Without the needed experience, it's going to be impossible for someone to tell they've made critical mistakes.
Yes you may design, but a good one requires a lot of effort. Following steps may be helpful
- Pick up a standard cipher
- Study its specifications (understand it well)
- Start studying Cryptanalysis of that cipher. You can start by studying following attacks on your chosen cipher (try to understand working of these attacks on your cipher and how your chosen cipher resist against these attacks)
Differential and Linear, Truncated Differential, Higher Order Differential, Boomrang, Algebraic, Interpolation, Related Keys, Slide
- uptil now, you should come to know about purpose behind function each component of the cipher. Why Sbox is there, what MDS is doing, whats the purpose of particular bit-wise-rotation, why key is being mixed before or after Sbox, How non linearity/ diffusion is being achieved, which component of cipher is providing resistance against what type of attack etc
- Now Pick up one component(Sbox, MDS, bit-wise-rotation etc) of the cipher, try to replace it with some component of your own and analyze the change in cipher security.
- If you think you did well in step 5, go to step 5 untill you have changed all the desired components, if not, go to step 3 and fill the missing blocks of required knowledge :)