Deterministic key selection using iterative value comparison - how dumb was this?

Question

Just to start, this is mainly a curiosity question that's been nagging me for a while. I did a fair bit of googling to see if someone has already performed a righteous expose of this silly idea, but couldn't find anything.

A friend and I were working together on a small project to learn the basics of practically implementing encryption. The details aren't particularly important, all we wanted it to do was share encrypted strings over an IP network.

We encountered an interesting problem when working out how to exchange a shared secret once an RSA handshake was performed. Since this tiny little program didn't work on the concept of client/server, we needed a way to ensure both ends would select the same key. Initially I investigated a pre-packaged implementation of ECC+Diffie-Hellman (from .NET), but there was a fatal flaw - the class only worked on Windows, but my friend is on a Mac.

As far as I know there aren't any other key-exchange implementations packaged in .NET, so I ended up coming up with a braindead-simple solution:

1. Both peers send a freshly-generated key (specifically, an AES key byte array) to the other.
2. Both peers feed their key and their partner's key into an algorithm, which iterates sequentially over the byte values (I think as implicitly converted 8-bit integers?) within both key arrays.
3. If one key has a greater value than the other at a certain index, that key is returned as the selected key.
4. If this condition is never met during the iteration, then the keys are identical and it doesn't matter which is returned. (IVs were present but ignored.)

This hackjob ten-line algorithm worked far better than it had any right to. With my limited understanding of the math involved, this makes sense computationally - there are $256^{32}$ possible keys and a $\frac{255}{256}$ chance of getting a result per (extremely cheap) comparison, so you're basically guaranteed an instant usable result.

However, I'm still curious - what does this algorithm look like from a security standpoint? If I've learned anything from browsing this SE, it's that even the simplest/most innocuous things to a layman like me can represent glaring security holes.

(Also, I swear this will never be used for something important. I know about Bouncy Castle now, thank God.)

Answer 1

Contrary to Diffie-Hellman, this does not resist passive eavesdropping. An adversary capturing the network traffic will get the "freshly-generated keys" that "both peers send" in step 1, and can apply the algorithm of steps 2/3/4 to find the shared key that ultimately encrypts the traffic that follows.