Is using only one or two test vector(s) for ChaCha20 enough for validation of coded algorithm?

Question

For validating the ChaCha20 encryption/decryption algorithm written in VB.NET, I am looking for more ChaCha20 test vectors that are based on the final spec for ChaCha20 that can be found here. See the test vector in sub-chapter 2.3.2. This test vector validates my code which provides the exact result specified in the test vector. (there is an almost identical vector later in on the document, two blocks with different counters, validates perfectly against that as well).

Test vectors that I found in draft ChaCha20 specifications do not give the exact same result as specified in these draft spec test vectors. They are using a different-size nonce compared to the final spec (draft 64 bits, final 96 bits) and the state of the counter is not specified (I assume it is 0).

Of course the question is if just one or two test vectors is enough to validate how an encryption algorithm is coded, my hunch is that more is very preferable but saying that as a true cryptography amateur.

It would be great to have more test vectors based on the final ChaCha20 spec to assure the code provides the exact results needed. Thank you!

    2.3.2.  Test Vector for the ChaCha20 Block Function

   For a test vector, we will use the following inputs to the ChaCha20
   block function:

   o  Key = 00:01:02:03:04:05:06:07:08:09:0a:0b:0c:0d:0e:0f:10:11:12:13:
      14:15:16:17:18:19:1a:1b:1c:1d:1e:1f.  The key is a sequence of
      octets with no particular structure before we copy it into the
      ChaCha state.

   o  Nonce = (00:00:00:09:00:00:00:4a:00:00:00:00)

   o  Block Count = 1.

   After setting up the ChaCha state, it looks like this:

   ChaCha state with the key setup.

       61707865  3320646e  79622d32  6b206574
       03020100  07060504  0b0a0908  0f0e0d0c
       13121110  17161514  1b1a1918  1f1e1d1c
       00000001  09000000  4a000000  00000000

   After running 20 rounds (10 column rounds interleaved with 10
   "diagonal rounds"), the ChaCha state looks like this:

   ChaCha state after 20 rounds

       837778ab  e238d763  a67ae21e  5950bb2f
       c4f2d0c7  fc62bb2f  8fa018fc  3f5ec7b7
       335271c2  f29489f3  eabda8fc  82e46ebd
       d19c12b4  b04e16de  9e83d0cb  4e3c50a2

   Finally, we add the original state to the result (simple vector or
   matrix addition), giving this:

   ChaCha state at the end of the ChaCha20 operation

       e4e7f110  15593bd1  1fdd0f50  c47120a3
       c7f4d1c7  0368c033  9aaa2204  4e6cd4c3
       466482d2  09aa9f07  05d7c214  a2028bd9
       d19c12b5  b94e16de  e883d0cb  4e3c50a2

Answer 1

I'd say that this is partly the case. Because of the rounds, even a single block operation will be operated on using multiple times. A block also doesn't take any special input: you have one block that is mapped to one output block.

However, that's not the complete story. First of all, some of the calculations in the rounds may require a bit more magic than usual. If there are any possible edge cases then those may not be triggered by one single block operation. Those you should test separately.

More importantly: while the above tests the block operation, it doesn't test the streaming operation. This is much more likely to cause issues. The problem is that encoding / decoding (e.g. bytes to words) and buffering are less trivial than that many seem to expect. So it makes sense to test all kinds of plaintext sizes, for instance, and make sure that update functions perform correctly. This is not so different from testing non-cryptographic functions; you'd expect the same issues with encoding/decoding operations and I/O for instance.

One issue that I've seen many times is that testing teams ignore the implementation itself. It is all fine to test against official test specifications, but if there are implementation specific design choices made, then these should be tested as well. As an example: it could well be that an implementation has a buffer overflow even if all official tests run without incident. I'll not repeat any horror stories here, but yeah...

Finally, as this is a cryptography site: if you make any claims about protection against side channel attacks then you should probably test those as well. Of course you're somewhat lucky here as Bernstein generally assumes these kind of attacks when designing a cipher - but implementations may be vulnerable none-the-less.

---

All this said, it never hurts to run your implementation against a more extensive set. Just take any existing library (or two independent ones) and generate a large set of vectors yourself. Again, focus on edge cases; doing a bit of fuzzy testing never hurts either.