It is unclear if you wanted to compare TLS 1.1 PRF or TLS 1.2 PRF. Different TLS versions have different PRFs. Assuming you meant TLS 1.1 PRF although you linked TLS 1.2 RFC.
TLS 1.1 PRF
Short: HKDF is commonly a better choice than TLS 1.1 PRF, but not always.
Consider these aspects:
- HKDF is a generic construct.
- HKDF is extract and expand.
- TLS1.1 PRF is just extract or expand.
- TLS1.1 PRF is intended for TLS protocol.
- HKDF can be used with hash functions chosen by the user.
- TLS1.1 PRF uses MD5 and SHA-1 hashes. Thus, it does not allow using stronger hash functions.
- TLS1.1 PRF uses two hash functions which causes extra complexity and complicates security analysis.
- TLS1.2 defined new PRF function to use. (So for purposes of TLS, TLS1.1 PRF has gotten deprecated).
TLS1.1 PRF has been analyzed in Hash function combiners in TLS and SSL. Based on this analysis, HKDF, when used with a strong hash function is likely stronger than TLS1.1 PRF.
Comparing HKDF with TLS 1.2's PRF.
NIST SP 800-108 Recommendation for Key Derivation Functions defines various ways to construct Key Derivation Function: - Counter Mode - Feedback Mode - Double-Pipeline Iteration Mode
For NIST, all these approaches are equally acceptable. The Feedback Mode matches HKDF's Expand phase. The Double-Pipeline Iteration Mode matches TLS 1.2 PRF. Thus we could assume, that TLS 1.2 PRF could be considered to be similar to HKDF's Expand phase if used with the same hash function underlying the hmac.
Purpose of HKDF's extract phase is quoted from RFC 5869:
In many applications, the input keying material is not necessarily distributed uniformly, and the attacker may have some partial knowledge about it (for example, a Diffie-Hellman value computed by a key exchange protocol) or even partial control of it (as in some entropy-gathering applications). Thus, the goal of the "extract" stage is to "concentrate" the possibly dispersed entropy of the input keying material into a short, but cryptographically strong, pseudorandom key. In some applications, the input may already be a good pseudorandom key; in these cases, the "extract" stage is not necessary, and the "expand" part can be used alone.
For applications where extract phase is beneficial (distribution of input keying material is not uniform), HKDF is certainly the way to go.
For other uses, where expand phase is used alone, any one of these two (or in some cases counter mode) from NIST SP 800-108 could be used. (Note: If you use TLS 1.2 PRF outside context of TLS protocol, maybe better define the function in terms of NIST SP 800-108, instead of TLS 1.2 PRF from the RFC).
- If you need extract use HKDF.
- If you need expand only, use functions from NIST SP 800-108.
- Generally you should not use TLS 1.1 PRF except for compatibility with TLS protocol.