I have made some reverse engineering on proprietary software and came up with some DHE algorithm that works like this:
First and foremost, the client and the server shares some public parameters (g, p) where p is a prime number :
# the client generates some random number X # Then... A = g^X % p B = 4^X % p
At this stade, A and B are two numbers computed with some exponentioation modulus p.
The client keeps A for itself and sends B over network to the server.
I can't guess out what's the server doing with B.
What comes up next is that the server sends the client back some buffer which is 20 bytes long. This buffer is xored with the first 20 bytes of the little-endian representation of A. Meaning something like this:
key = buffer ^ A[0...20]
And there you go, key is the shared key between the client and the server.
I am trying to guess out how the server generated that xoring buffer, and what did it compute with 'B'. Any clue?
For instance, I have these numbers:
X (random number) = 8788592173807205653697600075440308493757969005221433286958441247128089107594429784362875543428031794230308326680487467902465260209524609291268858491985100
How could the server compute A?