Bad custom encryption

Question

I hope you can help me understand the issue. I know creating my own encryption is not a thing which you should do, but I did this simple "encryption" a long time ago and discussed it with my teacher. He said that it is weak and should not be used in production, but he couldn't really explain why (or at least I didn't get it). Maybe one of you will be kind enough to help me.

The fundamental concept is to create an "endless" hash that is used to encrypt the whole plain text. I implemented the algorithm like:

1. A pre-shared secret and the plaintext are entered by the user
2. A hash is created from the secret
3. The plaintext is split into blocks with the size of the length of the hash
4. Each byte of the 1. block is aggregated to the according to byte of the hash (1. byte of block + 1. byte of hash = 1. byte of encrypted block, 2. byte of block + 2. byte of hash = 2. byte of encrypted block etc)
5. If there are more blocks to encrypt a new hash is generated from the last encrypted block + the last plain text block. This hash is used for encrypting the next plain text block (like step 4). This process repeats until all blocks are encrypted.
6. Last but not least all encrypted blocks are merged back again to one

As far as I understand, it should be secure as long as I use a secure hashing algorithm. Please help me understand why I am wrong.

You find a javascript implementation and example HTML page here: https://github.com/EdwardTivrusky/codebook

Answer 1

From the way I understand your question, your encryption scheme works as follows:

For a message consisting of $\ell$ blocks $m=m_1\|\dots\| m_\ell$ and a key $k$ the ciphertext $c=c_1\|\dots\| c_\ell$ is computed as $c_1:=m_1\oplus H(k)$ and $c_i:=m_i\oplus H(c_{i-1},m_{i-1})$ for $2\le i \le \ell$.

This scheme has several problems. First of all it's deterministic and therefore cannot be CPA secure, the basic minimum we generally require of an encryption scheme.

But it gets worse. In any standard definition of security, it is assumed that the attacker can choose, or at least know, parts of the plaintext. In your scheme, if the attacker knows the $n^{\text{th}}$ plaintext block $m_n$, they can compute $H(c_n,m_n)$ and thereby reveal the next plaintext block (and by extension the rest of the message).