What papers describe the rationale behind cryptography algorithms

Question

I am trying to introduce myself to cryptographic hashing and encryption. I am looking for original papers of sorts, to describe the inspiration behind why they choose certain numbers, if they are relevant, and why they choose certain algorithm features. But every paper I have found so far is like this which just describes the algorithm like encyclopedic metadata. Where can I find the meat on the bones, the why they chose certain features?

I assume starting at earlier algorithms (like SHA1) is probably better because they may specify the reasoning in more detail. But as they get to more recent (and secure) algorithms, they just cite the old algorithms for the meaning behind the decisions.

I am looking for things like SHA256 or other secure algorithms, but perhaps simpler nonsecure ones are better to start. For example, why does the SHA1 algorithm use 5 prime numbers? Why not 4, or 8, or non-primes?

Before processing any blocks, the H's are initialized as follows: in
   hex,
      H0 = 67452301
      H1 = EFCDAB89
      H2 = 98BADCFE
      H3 = 10325476
      H4 = C3D2E1F0.

That's all that is said.

Answer 1

First, I understand your question and the frustration that comes along with it, and it is not uncommon. I will warn you that from my perspective, there isn't really an easy answer that provides quick satisfaction. But let me try to point you in the right direction.

For a simple beginning path, I will point you to Jean-Philippe Aumasson's book, "Serious Cryptography" in Chapter 6 page 111 there is a section on "Building Hash Functions" but also notice that the section BEFORE this one talks about the characteristics of hash functions. If you read the "Building Hash Functions" section you won't really get the answer I think you are looking for asking (i.e. Why these constants?), however you will get some clues to constructions.

Looking into the characteristics of what hash functions need to provide (i.e pre-image resistance, 2nd pre-image resistance, and collision resistance) you see the goal that is trying to be achieved by a hash function. Until you study attacks on these goals, and deep security proofs of the functions, you will not come close to answering the questions you are asking. They are not easy question.

Two papers I will recommend. Bellare, Kilian and Rogaway "The Security of the Cipher Block Chaining Message Authentication Code" gives a formalism for MACs, PRFs and PRPs which is essential for your deeper understanding. Also Bellare, Canetti and Krawczyk "Pseudorandom Functions Revisited: The Cascade Construction and its Concrete Security" which discusses the iterative construction used in hash functions. These will give you some of the formal basis used in constructing hash functions (IMHO). Another paper that I believe is absolutely fundamental for going to this deeper level in cryptography is Bellare, Desai, Jokipii and Rogaway "A concrete security treatment of symmetric encryption"

Once you really consume these papers, and you get a handle on the simulation games used to prove security, and the attacks that you need to present, certain things will become more transparent. In addition, you will be able to read some proofs and attacks on algorithms with more clarity.

I would then recommend some reading on the Random Oracle Model, which is often used in proofs that use hash functions, and then look at papers around the SHA3 competition. I think it is really illuminating to read papers on SHA3 and thesis on hash functions - I will recommend one though I won't make any claim that it is the best (Ozgul Kucuk "Design and Analysis of Cryptographic Hash Functions" 2012).

These references should start you in your journey to acquire the knowledge you want so you can answer the questions you posed. I hope you find this direction useful.