ECDH anonymous key exchange to avoid PKI

Question

I want to use TLS to encrypt the communication between peers in a P2P network. Each peer has a well known 256bit peer identifier (the public key of a 256bit elliptic curve keypair). Both peers need to be authenticated against the ECC key.

Each peer has only one ECC key, no RSA key. During the key exchange step, I do not want each peer to generate a self-signed certificate with a bogus RSA key because that would render the authentication meaningless. The peers would have to blindly accept the self-signed RSA certificate because they would not be able to link the RSA key to the ECC key.

Since there is no PKI "authority" to sign each peer certificate, I am considering using ECDH in anon mode to perform the key exchange (TLS_ECDH_anon_WITH_AES_256_CBC_SHA). The anon mode of ECDH will not authenticate the peers at the TLS level, so I want to do the authentication at the application level, by sending/answering a challenge signed by the well known ECC key.

Is this a proper use of ECDH anon, and will a simple challenge/response scheme at the application level guarantee peer authentication?

Also, will this scheme provide forward secrecy?

Answer 1

I have figured out a way to use TLS with only a EC key by using DSA instead of RSA. I had not realized you could do DSA with a EC key. My mistake was trying to use RSA to sign the certificate.

Now I can generate my certificate and self-sign it only with the EC key. The peer will simply check to make sure the certificate was signed by the appropriate node ID (which is the same as the EC public key). And that the public key from the certificate is the same as the node ID.

In practice it simply boils down to using the cipher suite TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 with a custom trust manager.

Thank you both for taking time to answer my question.

Answer 2

First, one point.

Each peer has a well known 256bit peer identifier (the public key of a 256bit elliptic curve keypair).

Therefore, you have PKI, even though "there is no PKI 'authority'".

Is this a proper use of ECDH anon, and will a simple challenge/response
scheme at the application level guarantee peer authentication?

Yes, and No.
An adversary could mitm the the ECDH anon and then just forward the application level data.

Also, will this scheme provide forward secrecy?

It won't provide forward secrecy against a fully active adversary, due to the previous sentence.
It will provide forward secrecy against an adversary that does not tamper with the DH anon.

Do the DH anon, and derive one more key than will be used for continuing exchange of data.
Call the extra key Ktmp, and make the encryption using that key be authenticated encryption.
The initiator encrypts an ordered pair whose entries are its identity and its
signature on [the bit $0$ concatenated with all of the public data from the DH anon,
separated as may be necessary]] with Ktmp, and sends the ciphertext to the responder.
The responder verifies and decrypts that ciphertext with Ktmp, checks that the identity is
acceptable, and verifies the signature. $\:$ If any of those steps fails then the responder aborts.
The responder encrypts an initial bit concatenated with an ordered pair whose entries are its identity
and it's signature on [the bit $1$ concatenated with all of the public data from the DH anon, separated
as may be necessary] with Ktmp, where the choice of the initial bit indicates that it will or won't
immediately send encrypted application-level data, and sends the ciphertext to the responder.
The initiator verifies and decrypts that ciphertext with Ktmp, checks that the
identity is acceptable, and verifies the signature. $\:$ If the initial bit of the
plaintext was $1$, then the initiator verifies and decrypts the following
encrypted application-level data. $\:$ If any of those steps fails then the initiator aborts.
The initiator encrypts $1$ or $0$ with Ktmp, where the choice of $1$ or $0$ indicates
that it will or won't immediately send encrypted application-level data,
erases Ktmp, and then sends the ciphertext to the responder.
The responder verifies and decrypts that ciphertext with Ktmp.
If the plaintext was $1$, then the responder verifies and decrypts the following
encrypted application-level data. If either of those steps fails then the responder aborts.
Regardless of whether or not either of those steps failed, the responder erases Ktmp.