After some more searching, I came across RFC 8448, which was actually referenced in RFC 8446, and which contains test vectors for the key derivation and handshake messages.
Another item of interest, I came across ACVP, which seems helpful, but is a hosted service that requires explicit permission to use.