Well, the multiplicative inverse of $a$ is defined to be that value $b$ for which $a \times b = 1$, where $\times$ is the multiplication operation in the field/ring/group in question.
Because we're talking about the group of multiplication modulo 65537, that means that the problem is, given $a$, find $b$ such that $ab \bmod 65537 = 1$.
Now, the % operator is C doesn't do it. The classical way is to use the Extended Euclidean algorithm, where the two inputs to the algorithm is $a$ and $65537$; and have it find a solution to the equation $ax + 65537y = GCD(a, 65537) = 1$; the value $x$ is the multiplicative inverse you're looking for.
Of course, since there are only 65536 possible inverses, another possibility is simply have a table of the 65536 possible inverses, and just do a lookup. In that case, you can use the Extended Euclidean algorithm to build the table.
Oh, and as a reminder; idea interprets the 0000 bit pattern as the value 65536 as far as multiplication is concerned (as the value 0 doesn't have an inverse).
CF80
in hexadecimal is 53120 in decimal, but not 3080 by any stretch of imagination; that is -3080 modulo 65536, or -3081 modulo 65537; also, 3080 converted to hexadecimal is0C08
, not3080
. In the context you want unsigned numbers, and displaying them as such (in decimal or hexadecimal). Most importantly, the multiplicative inverse of $x$ is $y$ such that $x⋅y=1$ in the multiplicative group $\pmod{65537}$ (with 65536 mapped to the 16-bit value0x0000
), and you make absolutely no step towards computing that $y$ when $x$ is 53120 or hexadecimalCF80
, your goal. $\endgroup$ – fgrieu Mar 15 '13 at 7:37