I'm currently reverse engineering a program that uses Curve25519 key exchange in network communication. I have only a basic understanding of ECC, so maybe this thing just seems strange to me.

The client generates a private key, derives the public key from it, and sends it to the server. So far, that seems like a normal key exchange for me.

However, the server does not generate a simple public key and sends it to the client. The server generates a key pair and sends the public part to the client.

The client uses this public key and calculates a strange mechanism:

(Uses 25519ref10 - simplified pseudocode)

// The public key the server send to the client
ge_p3 pubkey = ge_frombytes_negate_vartime(THE RECEIVED PUBLIC KEY FROM SERVER);

// Scalarmult the sha512 with the public key from server.
// This seems very strange to me... but its getting stranger.
ge_p1p1 res = ge_scalarmult(THE SHA512 OF THE PUBLICKEY FROM THE SERVER + 8 EXTRA BYTES, pubkey);

// Okay
ge_p3 r = ge_p1p1_to_p3(res)

// This is a hardcoded byte constant in the client binaries.
// The private part of this can also be found in the server binaries.
ge_p3 parkey = ge_frombytes_negate_vartime(A CONST PUBLICKEY);
pargrp = ge_p3_to_cached(parkey);

// Now this parkey const is getting added to the calculated scalarmult from hash & pubkey
ge_p1p1 a = ge_add(r, pargrp);
ge_p3 r2 = ge_p1p1_to_p3(a);

// r2 will now converted to a byte array and the last byte is XOR'd with 0x80
result = ge_p3_tobytes(r2);
result[31] ^= 0x80;

This result is now used as the server public key.

So the server has to know the private key of "result" (and indeed, while this exchanging is happening, the private key of this calculation is in the memory of the server). But I cannot reverse this part of the calculation from the server binaries.

The only thing that I know, is that the public key is scalar (a hash of public key, public key) + const. Of course the server cannot calculate the private key from this special publickey. Does anyone have any idea how this private key could be generated?

  • $\begingroup$ Note that your question in the title is how to generate the public key , and below is the question about the private key. You're not explicitly asking to identify the protocol that is used. Finally, if you provide references then it may be more easy for others to help you. $\endgroup$ – Maarten Bodewes Jul 12 '18 at 17:09
  • $\begingroup$ Oh yes you're right. The questions aims on the generating of this private key. $\endgroup$ – BuddyYuz Jul 12 '18 at 17:35
  • $\begingroup$ From the question it is not clear, what all those transformations actually do. However, all those transformations are most likely easy to invert - and are completely irrelevant for security. Also, it's quite irrelevant to include the hash of the public key, because that's public. Also irrelevant: The bitflip in the last line of code doesn't add anything. It seems, this is actually just a regular ECDH key exchange with a 64 bit secret value (called EXTRA BYTES) - with quite a bit of additional code, which just pretends to add complexity - it does nothing from a security standpoint. $\endgroup$ – tylo Jul 13 '18 at 13:53
  • $\begingroup$ The biggest problem here: Using a 64 bit secret value is absolutely terrible and should not be used. $\endgroup$ – tylo Jul 13 '18 at 13:57
  • $\begingroup$ Well it's not my fault, i just reverse engineering this masterpiece of security. $\endgroup$ – BuddyYuz Jul 13 '18 at 21:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.