How is EC key encoded in PKCS#8?

Question

I just started working with certificates and signatures. For an application I write I need a key pair for ECDSA signatures, using the elliptic curve secp384r1 (aka NIST P-384).

I produced such a key pair with the Bouncy Castle lib in Java. Just for testing purposes I then used the JcaPEMWriter to write the two keys to files.

I got the following results:

-----BEGIN EC PRIVATE KEY----- MDUCAQEEMB4Tr3coGQbhIax2Aqs3IJW3D2QmsG1efvhrCeY+6F0K9fTAFT1+mTyL ENJ2lUdD/Q==
-----END EC PRIVATE KEY-----

and

-----BEGIN PUBLIC KEY----- MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEhUvTJddWZidioBHCnPk23SH2C1lITFEK UGH63OqRcbE3cfGkykx0XUdHh8ZIKMUhikKr8+ln59sh7Tz1D6TqjtNPsuEMO07t ykzbQM+xZT0DMwnVNx+B0UT1Rr9JrNVX
-----END PUBLIC KEY-----

My questions are:

1) Why is the format of the two key representations different? The private key is in PKCS#8, the public key in X.509 format (at least that is what Java's "getFormat" method tells me). I think there must be some good reason why the developers decided for these different representations, but for me it is just confusing ;-). Is this due to some standard?

2) Next I went to https://lapo.it/asn1js to decode the keys. For the public key I got some information about the algorithm used, and then a 776 bit string. These are 97 bytes. If I understand it correctly, those are made up of one byte telling me that the elliptic curve point is in uncompressed encoding, 48 bytes (=384 bit) for the x-coordinate and 48 bytes for the y-coordinate.

But when I try to decode the private key in the same manner, I get only 48 bytes for the actual key. What exactly do these bytes represent? Is this the curve point in it's compressed form? But in this case, shouldn't there be a flag at the beginning telling me that this is compressed form?

Maybe you can help me get a better understanding of what's going on here. Thank you in advance!

Answer 1

But when I try to decode the private key in the same manner, I get only 48 bytes for the actual key. What exactly do these bytes represent? Is this the curve point in it's compressed form? But in this case, shouldn't there be a flag at the beginning telling me that this is compressed form?

When we generate an EC public/private key pair, we pick a number $x$ and compute the elliptic curve point $xG$, which is $G$ (the well-known "generator point") added to itself $x$ times.

The public key is the point $xG$; because it is a point, we need to state whether we're expressing that point in compressed or uncompressed format.

In contrast, the private key is just the value $x$; this is just an integer, and so doesn't need to be 'compressed'.

As for why your $x$ is 48 bytes (or about 384 bits), we try to select the integer randomly between 1 and the size of the group (which is, in your case, circa $2^{384}$); selecting from a smaller range would reduce the security we get, and selecting from a larger range wouldn't help.

Answer 2

Not really an answer: only a minor point but explicating clearly is much too long for comments.

The encoding you get from Java [anypkalg]PublicKey.getEncoded() is in a format Java calls "X.509" and is more exactly the SubjectPublicKeyInfo structure from X.509 and PKIX. This is designed to handle multiple algorithms, and encodes both an AlgorithmIdentifier that identifies the algorithm and parameters -- here id-ecPublicKey to identify this key as Elliptic Curve (in X9.62 format) and the Object Identifier for secp384r1 to identify the curve used -- plus a BIT STRING containing the actual key in an algorithm-dependent format, which for this ECC curve in uncompressed format is indeed 97 octets: 1 octet with value 04 meaning uncompressed, then 48 octets for the X coordinate then 48 octets for the Y coordinate. Specific algorithms are detailed in rfc3279 and rfc5480 among others.

Java [anyalg]PrivateKey.getEncoded() similarly is in PKCS8 unencrypted which similarly handles multiple algorithms by combining an AlgorithmIdentifier with algorithm-specific data. However, BouncyCastle JacPEMWriter does NOT use PKCS8 for a privatekey; the format you have for your privatekey is the OpenSSL-defined 'traditional' or 'legacy' format for ECC, which is that defined by SECG document SEC1 which besides a version is only required to contain the actual private number, but can optionally contain the curve id and/or public point, and it would be nice if Bouncy preserved the curve id here where it is useful (unlike PKCS8 where it is already in the algid). PKCS8 format has PEM type PRIVATE KEY or ENCRYPTED PRIVATE KEY, NOT EC PRIVATE KEY or any other [algorithm] PRIVATE KEY; to create that with Bouncy use org.bouncycastle.openssl.PKCS8Generator and the lower-level org.bouncycastle.util.io.pem.PemWriter (note Pem not PEM). Alternatively for unencrypted take the PKCS8 binary from key.getEncoded(), convert to base64 with line breaks (standard in java.util in 8+) and add the BEGIN/END lines.

Why different standards are used for public and private keys is because public keys are supposed to be passed from one system to another or many others, and the public certificate format was quickly (1988) developed to do this, while private keys are supposed to be kept private and thus there was not perceived so urgent or immediate a need to standardize them. Why there are still multiple standards for private keys in OpenSSL (and lots of other things that use or are made compatible with OpenSSL) is history; PKCS8 hadn't yet been adopted when Eric Young began developing SSLeay which evolved into OpenSSL. It was added fairly soon, IIRC before 2000, but there were already users who had and wanted to keep files in then-existing 'traditional' formats, so until OpenSSL 1.0.0 in 2010 it described PKCS8 as 'new' and optional -- though preferable, especially when encrypted. (The password-based key derivation used by OpenSSL for 'traditional' format files is much weaker than that used for PKCS8 files; there are many existing Qs on this here and in security.SX, and I think stackoverflow also, you should be able to find, if not I will add them.)