Varying ECDH key output on different ECC implementations on the secp256k1 curve

Question

Like the title says, I have used several implementations of ECDH, and I have found a discrepancy with libraries that wrap libsecp256k1. It's likely the case that I'm making a mistake but I cannot see what it is. I display all the public keys for a sanity check and to make sure that I'm not giving in input in the incorrect way. I could be invoking ECDH incorrectly, but I think I am. There could be something else that I can't don't see (which I will feel very silly about), but it is the point of me asking here. If anyone can find any issues please let me know.

Using secp256k1 (0.13.2) [wrapper for libsecp256k1] for python 3.6.4

import secp256k1

privkey1 = "82fc9947e878fc7ed01c6c310688603f0a41c8e8704e5b990e8388343b0fd465"
privkey2 = "5f706787ac72c1080275c1f398640fb07e9da0b124ae9734b28b8d0f01eda586"
aa = secp256k1.PrivateKey(bytes.fromhex(privkey1), raw=True)
bb = secp256k1.PrivateKey(bytes.fromhex(privkey2), raw=True)

cc = aa.pubkey.ecdh(bb.private_key).hex()
print("pub1:{}".format(aa.pubkey.serialize(compressed=False).hex()))
print("pub2:{}".format(bb.pubkey.serialize(compressed=False).hex()))
print("ecdh:{}".format(cc))

Output

pub1:04c7c674a223661faefed8515febacc411c0f3569c10c65ec86cdce4d85c7ea26
     c617f0cf19ce0b10686501e57af1a7002282fefa52845be2267d1f4d7af322974
pub2:04b80cdf1422644ccfb0a2c73103bdfa3cc96786c3e63d8df70267fc7fffe711a
     1000d37a20cefd1fdcceec0b0b3f25a46c8a430800ba0c19f4ae0cfc582de8fb8
ecdh:5935d0476af9df2998efb60383adf2ff23bc928322cfbb738fca88e49d557d7e

Using crypto (builtin) in nodeJS v8.2.1 (uses openssl under the hood)

const crypto = require('crypto');

const privkey1 = "82fc9947e878fc7ed01c6c310688603f0a41c8e8704e5b990e8388343b0fd465"
const privkey2 = "5f706787ac72c1080275c1f398640fb07e9da0b124ae9734b28b8d0f01eda586"

const aa = crypto.createECDH("secp256k1");
const bb = crypto.createECDH("secp256k1");
aa.setPrivateKey(privkey1, "hex")
bb.setPrivateKey(privkey2, "hex")
const cc = aa.computeSecret(bb.getPublicKey())
console.log(`pub1:${aa.getPublicKey('hex')}`)
console.log(`pub2:${bb.getPublicKey('hex')}`)
console.log(`ecdh:${cc.toString('hex')}`)

Output

pub1:04c7c674a223661faefed8515febacc411c0f3569c10c65ec86cdce4d85c7ea26
     c617f0cf19ce0b10686501e57af1a7002282fefa52845be2267d1f4d7af322974
pub2:04b80cdf1422644ccfb0a2c73103bdfa3cc96786c3e63d8df70267fc7fffe711a
     1000d37a20cefd1fdcceec0b0b3f25a46c8a430800ba0c19f4ae0cfc582de8fb8
ecdh:3a17fe5fa33c4f2c7e61799a65061214913f39bfcbee178ab351493d5ee17b2f

Using elliptic v6.4.0 in nodeJS v8.2.1

const EC = require('elliptic').ec;
const ec = new EC('secp256k1');

const privkey1 = "82fc9947e878fc7ed01c6c310688603f0a41c8e8704e5b990e8388343b0fd465"
const privkey2 = "5f706787ac72c1080275c1f398640fb07e9da0b124ae9734b28b8d0f01eda586"

const aa = ec.keyFromPrivate(privkey1,"hex");
const bb = ec.keyFromPrivate(privkey2,"hex");

const cc = aa.derive(bb.getPublic());
console.log(`pub1:${aa.getPublic('hex')}`)
console.log(`pub2:${bb.getPublic('hex')}`)
console.log(`ecdh:${cc.toString('hex')}`)

Output

pub1:04c7c674a223661faefed8515febacc411c0f3569c10c65ec86cdce4d85c7ea26
     c617f0cf19ce0b10686501e57af1a7002282fefa52845be2267d1f4d7af322974
pub2:04b80cdf1422644ccfb0a2c73103bdfa3cc96786c3e63d8df70267fc7fffe711a
     1000d37a20cefd1fdcceec0b0b3f25a46c8a430800ba0c19f4ae0cfc582de8fb8
ecdh:3a17fe5fa33c4f2c7e61799a65061214913f39bfcbee178ab351493d5ee17b2f

Using secp256k1 (3.5.0) [wrapper for libsecp256k1] for nodeJS v8.2.1

const secp256k1 = require('secp256k1')

const privkey1 = "82fc9947e878fc7ed01c6c310688603f0a41c8e8704e5b990e8388343b0fd465"
const privkey2 = "5f706787ac72c1080275c1f398640fb07e9da0b124ae9734b28b8d0f01eda586"

const privkey1buf = Buffer.from(privkey1,"hex")
const privkey2buf = Buffer.from(privkey2,"hex")
const pubkey1 = secp256k1.publicKeyCreate(privkey1buf,false)
const pubkey2 = secp256k1.publicKeyCreate(privkey2buf,false)

const cc = secp256k1.ecdh(pubkey1, privkey2buf)
console.log(`pub1:${pubkey1.toString('hex')}`)
console.log(`pub2:${pubkey2.toString('hex')}`)
console.log(`ecdh:${cc.toString('hex')}`)

Output

pub1:04c7c674a223661faefed8515febacc411c0f3569c10c65ec86cdce4d85c7ea26
     c617f0cf19ce0b10686501e57af1a7002282fefa52845be2267d1f4d7af322974
pub2:04b80cdf1422644ccfb0a2c73103bdfa3cc96786c3e63d8df70267fc7fffe711a
     1000d37a20cefd1fdcceec0b0b3f25a46c8a430800ba0c19f4ae0cfc582de8fb8
ecdh:5935d0476af9df2998efb60383adf2ff23bc928322cfbb738fca88e49d557d7e

Using Bouncy Castle v1.8.1.3 in dotnet v2.1.4 (C#) here

Output

pub1:04c7c674a223661faefed8515febacc411c0f3569c10c65ec86cdce4d85c7ea26
     c617f0cf19ce0b10686501e57af1a7002282fefa52845be2267d1f4d7af322974
pub2:04b80cdf1422644ccfb0a2c73103bdfa3cc96786c3e63d8df70267fc7fffe711a
     1000d37a20cefd1fdcceec0b0b3f25a46c8a430800ba0c19f4ae0cfc582de8fb8
ecdh:3a17fe5fa33c4f2c7e61799a65061214913f39bfcbee178ab351493d5ee17b2f

Answer 1

The secp256k1 wrapper for Python invokes the secp256k1_ecdh() function from libsecp256k1 (from here). That C function not only computes the ECDH point, but explicitly hashes it with SHA-256. Indeed, the ECDH computation results in point $(x,y)$ such that:

x = 3a17fe5fa33c4f2c7e61799a65061214913f39bfcbee178ab351493d5ee17b2f
y = 14039097d4436cc600b6af5032afa00d30e2a090f00a6c52de374db8cff0e277

libsecp256k1 encodes the point in compressed representation: first byte is 0x03 (in compressed representation, first byte is 0x02 or 0x03, depending on the last bit of $y$), followed by the $x$ coordinate. Indeed, if you hash with SHA-256 the following sequence:

033a17fe5fa33c4f2c7e61799a65061214913f39bfcbee178ab351493d5ee17b2f

then you get:

5935d0476af9df2998efb60383adf2ff23bc928322cfbb738fca88e49d557d7e

Thus, you have two different conventions:

• The libsecp256k1 wrappers apply a SHA-256 hash on the resulting point (the complete compressed point, not just the $x$ coordinate).
• The other implementations return the $x$ coordinate itself.

We may note that, in general, only the final $x$ coordinate is used in standard ECDH (as defined by ANSI X9.63). What libsecp256k1 does is thus "non-standard" (but the Bitcoin world has never cared much about standards anyway). This also means that what the non-libsecp256k1 implementations return (the $x$ coordinate of the point) is not sufficient to compute the value returned by libsecp256k1 (since it lacks the information about the least significant bit of $y$); and what libsecp256k1 returns is not sufficient to compute the $x$ coordinate as returned by other implementations.