I am trying to create a DTLS client finished packet, where I need to generate verify data for handshake hash is need. And here I want to know how to generate handshake hash. As of now I am doing like this:

memcpy(hs_msg, client_hello_with_cookie+13, client_hello_with_cookie_len-13);
len = client_hello_with_cookie_len-13;
memcpy(hs_msg+len, server_hello+13, server_hello_len-13);
len += (server_hello_len-13);
memcpy(hs_msg+len, server_hello_done+13, server_hello_done_len-13);
len += (server_hello_done_len-13);
memcpy(hs_msg+len, client_key_exchange+13, client_key_exchange_len-13);

But the hash of handshakes seems to be wrong while comparing with openssl. (I am using sha1).

Need help to solve this...

Thank you very much.

  • $\begingroup$ it is still not clear can any one explain me how to create a finished message in dtls (client), i am using PSK_WITH_AES_128_CBC_SHA $\endgroup$ Jul 8, 2014 at 5:56

1 Answer 1


First, I am assuming, per https://security.stackexchange.com/questions/29172/what-changed-between-tls-and-dtls, that the client handshake protocol in DTLS is not different from that in TLS over TCP. This seems a safe bet since the client/server encrypted handshake protocol in OpenVPN's UDP implementation is the same as in standard TLS over TCP.

I am not sure what you mean by "using sha1", since the client encrypted handshake message hash $V$ use a concatenation of MD5 and SHA1 hashes of the concatenated handshake (TLS opcode 0x22) messages. Details are in Davies' "Implementing SSL/TLS," pp. 345-350:

$$V= \text{PRF}(\text{master secret},\text{"client finished"},\text{MD5}(H)\|\text{SHA-1}(H),12)$$

$H$ is the concatenation of the handshake messages; "PRF" is TLS-PRF, a Python implementation of which follows:

def TLS_PRF(secret,label,seed,length):
    def P_HASH(secret,seed,length,h):
            i = 1
            A = [seed]
            out = ''
            while len(out) < length:
                    out += HMAC(A[i]+seed,secret,hasher=h)
                    i += 1
            return out[:length]
    md5 = P_HASH(secret[:len(secret)/2],label+seed,length,hashlib.md5)
    sha1= P_HASH(secret[len(secret)/2:],label+seed,length,hashlib.sha1)
    for m,s in zip(md5,sha1):
            f += "%c"%(ord(m)^ord(s))
    return f 

If you run the concatenated 0x22 messages through the complicated process noted above, you should get agreement with at least the hash in the client's plaintext EHM.

Calculation of the server EHM hash is slightly trickier. To $H$, the extant concatenated 0x22 messages, you will need to concatenate the decrypted 12 byte value $V$ contained in the client EHM. Do not forget, per the RFC, to prepend this 12 byte value with the message type (probably 0x14) and three byte length value (0xc) - so add 14 00 00 0c <12 bytes of V> to the $H$ used for the client EHM, run that through the PRF (using "server finished" for your label), and you should have your final hash value.

Calculation of DTLS Application Data MACs

This is in response to an additional question posed by OP in comments to this answer, concerning calculation of the Application Data (DTLS packet type 0x17) MAC values. According to RFC 4347:

The DTLS MAC is the same as that of TLS 1.1. However, rather than using TLS's implicit sequence number, the sequence number used to compute the MAC is the 64-bit value formed by concatenating the epoch and the sequence number in the order they appear on the wire. Note that the DTLS epoch + sequence number is the same length as the TLS sequence number.

Here is some Python code that implements this for a packet at the beginning of the DTLS session, with the starting SeqNum and EpochNum values. $D$ is the decrypted application data including the "decrypted" IV at the start of the message, and the MAC and padding at end of message (basing all of on this on how Wireshark dissects DTLS traffic).

    SeqNum="000000000001".decode("hex")    #passed over wire 
    EpochNum="0001".decode("hex")          #passed over wire
    PacketType="17".decode("hex")          #Application Data
    ProtocolVersion="feff".decode("hex")   #DTLS 1.0
    PacketLength="0400".decode("hex")      #Tricky - the length of AppData (as passed in clear over the wire) minus 0x20 bytes of SHA-1 MAC and padding
    D="0xf00ba..."                         #The decrypted blob

Then the HMAC will be:


Don't forget the exclude the first 0x10 bytes (which will be the meaningless, "decrypted" IV) and the last 0x20 bytes (MAC+padding). Apart from not including/ordering EpochNum and SeqNum correctly, the only other big pitfall, as previously noted, is forgetting to adjust the PacketLength to reflect the fact that the MAC obviously doesn't include itself as part of the message (see Davies, p. 376 for a discussion of this point in traditional TLS context).

  • $\begingroup$ thanks for your reply, actually i am acting like client to openssl and trying to send client_key_exchange + ccs + encrypted_finished messages. In the reply I am not receiving any thing I means openssl is not sending reply. What I am doing is after generating 12 bytes of verify_data V creating pkt like: ) P = type(0x14)+len(3bytes)+msg_seq(2bytes)+frag_offset+frag_len(3bytes)+V; hmac_sha1(client_write_mac, P, digest); finished = Record+AES128(P+digest); Is that correct, if not how to create finished message. thanks. $\endgroup$ Jul 3, 2014 at 11:43
  • $\begingroup$ I apologize for misunderstanding your question. I don't have ready access to a computer for a few days, but haven't poked into this particular issue before, thank you for the opportunity to learn more. Once I get a computer again can work on this, but in meantime I assume that you have consulted stackoverflow.com/questions/23596622/… and it hasn't helped? The structure described in that message seems to be different from what you outlined above. If that's how you're doing it and still having problems can look at this in a few days. $\endgroup$
    – bjsdaiyu
    Jul 4, 2014 at 21:04
  • $\begingroup$ it is still not clear can you explain how to create a finished message in dtls, i am using PSK_WITH_AES_128_CBC_SHA thanks $\endgroup$ Jul 8, 2014 at 5:55
  • $\begingroup$ I know. I'm going to build a toy DTLS app and try to help you figure things out. Thanks for telling me the symmetric algorithm you're using, might as well cut that variable out. $\endgroup$
    – bjsdaiyu
    Jul 8, 2014 at 20:48
  • 1
    $\begingroup$ I solved application MAC issue also, the problem is i am not passing correct length of application data. $\endgroup$ Jul 14, 2014 at 5:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.