Signing with RSA-SHA256 algorithm using EC keys

Is it practical to sign a data using RSA-SHA256 algorithm with EC keys?

I used a small NodeJS script and crypto module (uses openSSL internally) to test this and I could successfully sign the data and verify the signature.

Pseudo-code:

Sign( data, EC-Key-p256, 'RSA-SHA256')

Verify( data, signature,public-key, 'RSA-SHA256')

Have used openSSL to create EC keys (p256v1) and used it for my testing.

• No, you shouldn't be able to actually use EC keys to produce an "RSA signature". I suppose that in this case the library is smart enough to see that the key is an EC key and that you want to sign using SHA256 and thus it just calls ECDSA with SHA256 instead.
– SEJPM
Sep 6 '18 at 22:04
• @SEJPM+ Yes (fixed) node-jwa ignores it, as answered in this dupe by The Two-Faced Bear. PS: the name is prime256v1 from X9.62 or P-256 from NIST FIPS 186 (used by rfc7518) or secp256r1 from SECG, but not p256v1. See Stallings and xkcd about standards. Sep 7 '18 at 3:29

tl;dr - RSA keys and ECDSA keys are not compatible and cannot be used in one anothers signature algorithms

In RSA you have the following:

• A public key $(e, N)$ where $e$ and $N$ are integers.
• A private key $d$ where $d$ is an integer and $ed \equiv 1 \bmod \phi(N)$, $\phi$ being Euler's totient function.

In ECDSA you have:

• A public key $Q$ which is a point on an elliptic curve e.g. $Q = (q_x, q_y)$.
• A private key $d$ which is an integer such that $dG = Q$, where $G$ is the base point of the elliptic curve.

Given those facts the question you pose doesn't make sense. Indeed an RSA private key and an ECDSA private key seem similar in that they are both integers, but in RSA $d$ needs to also have a corresponding $N$, which is usually left out since it is a public value (if you want to be pedantic the private key is really $(d, N)$ in RSA).

So say I generate an ECDSA private key $d$ and want to sign a message $m$ with it via RSA-SHA256. RSA-SHA256 would look something like $signature = \mathrm{sha256}(m)^d \bmod{N}$ (actually you would want to apply padding after the hash and before the exponentiation but that's not really important here).

But if only $d$ is generated, as one would do for an ECDSA private key, how does one perform the $\bmod{N}$ operation? You can't because $d$ by itself is an incomplete RSA private key, you also need the $N$ corresponding to the public key.

Let's say you do generate some $N$ after the fact so that you can perform signing. Verification would run into the same problem of keys not being compatible. RSA-256 signature verification for message $m$ and signature $s$ looks something like $\mathrm{sha256}(m) =^? s^e \bmod N$ where $(e, N)$ is the RSA public key. But if one does not have $(e, N)$ and is provided only a point $Q = (q_x, q_y)$ that lies on an elliptic curve, how does one then verify the signature? One of course can't.

SEJPM is probably correct in his comments that the library you're using sees that the signature algorithm doesn't match the key and chooses the correct signature algorithm for signing. If the library isn't raising a warning and quietly acting as though all parameters are valid I would consider switching libraries.