When defining protocol compliant with NIST SP 800-108, you just need to pick suitable options, which work well with your protocol.
If there is a need to be compatible with a specific pre-existing protocol, you may want to take a look at NIST SP 800-135Rev1, which defines application specific key derivation functions. It is notable to recognize that some of the NIST SP 800-135Rev1 key derivation functions are valid NIST SP 800-108, with specific options selected.
The best way to identify the KDF is to actually list all the applicable options from NIST SP 800-108 KBKDF. (What of three functions to use (CTR, FB, Pipeline); order of components, what is the content of label, and if the label is terminated by zero etc.) It is likely best to illustrate the protocol specification with clear examples which show the correspondence of NIST SP 800-108 functions (or alternative show invocations of underlying MAC function) as used in key derivation.
Testing NIST SP 800-108 implementation
The recommended way to test is via CAVP (Cryptographic Algorithm Validation Program). Testing takes a long time and will be performed by a CST (Certified testing Laboratory), which costs money. But you get to choose pretty specifically what to test (from reasonably large option space) and if you pass the testing, you will get a algorithm validation certificate.
For most common options, less expensive and quicker way could be to test against pre-existing unofficial test vectors from NIST/CAVP.
If the implementation needs not only to be FIPS compliant, but also is required to have US government certification (FIPS 140-2), then algorithm certificates for the applicable algorithms are needed.
If you need a set of test vectors for your KBKDF "instantiation", one way to do it is to first ensure correctness of implementation using one of the above mechanisms, and the use that "appears to be correct" implementation to generate the official test vectors, and attach the test vectors with the protocol specification.
The way to determine order, sizes of elements etc. used in the test vectors is explained in KBKDFVS (KBKDF Validation System).
The parameters KBKDFVS uses for Feedback Mode are (extracted from KBKDFVS):
- Pseudo-Random Function supported by the IUT
- List CMAC AES 128, 192, 256
- CMAC TDES 2, 3
- HMAC SHA1, SHA224, SHA256, SHA384, SHA512
- L-length of the derived keying material K0 in bytes – Enter all that apply:
- A minimum and maximum value for full block lengths supported
- A minimum and maximum value for partial block lengths supported
- Method(s) supported to generate K:
- an Approved RNG
- N/A – Out of the scope of the algorithm implementation
- Does the IUT support the counter being used as an input? If yes, what is the length of the binary representation of the counter i.
- 8 bits.
- 16 bits.
- 24 bits.
- 32 bits.
- Does the IUT support the counter coming before and/or after the iteration variable and/or after the fixed input data?
The information for other modes (i.e.) counter and double pipeline is similar.
This information is not sufficient to encode all that can be known about the KDF function, but is already a good start.
Finally addressing your questions
- Identify the KDF by specifying options from NIST SP 800-108. You should also have examples (and possibly test vectors), which allow correct implementation.
- The name is of NIST SP 800-108 and the name (or section) of the mode used.
- I'm also not aware of OID/ASN.1 applicable. (But maybe somebody has defined something for their implementations of NIST SP 800-108. There are no OID/ASN.1 which covers all the possible options of NIST SP 800-108.)
- If you need OID or ASN.1 module for your protocol, it is likely easiest for now to define your own.
- Implementations of SP 800-108 likely can be tested using CAVP or unofficial CAVP test vectors.
- KBKDFVS can be good example to see how NIST specifies the parameters in context of test vectors. Maybe you can use similar approach and terminology for the protocol?
Side Note: Depending on how you use HKDF, you are often using either NIST SP 800-108 or NIST SP 800-56C: HKDF "expand only" can be considered to be a variant of NIST SP 800-108 KBKDF. NIST SP 800-56C is basically attempt to define full HKDF in NIST terminology (and add AES-CMAC).