Why do web-services tend to use preshared secret keys for client authentication instead of public keys?

Question

Every API that I have ever signed up with gives you a secret key that you can then paste into your application. They know the key, and so do you. (The one exception may be VAPID for Web Push).

Why is this the case? Some apps and protocols already give users a public/private key pair, and store only the public keys in the database row for the user. The user then signs requests with their corresponding private keys, which are never supposed to leave the device. (Nevermind that in Web3 wallets, you can typically export a private key, I think this is a bad design decision that enables all kinds of phishing and scams.)

Now, I understand that for encryption, we do need to send a symmetric key alongside a payload, to decrypt it. But for simply signing a payload, wouldn't we be able to just hash it and sign the hash with the private key?

The only rationales for using symmetric keys I can think of, have terrible flaws:

• Agency. Since the platform is doing the work anyway, then what's the harm in it also knowing the symmetric key? Well, there are two problems with this. One is that if the platform's database is compromised, all the secrets are leaked. Second is that the platform may impersonate the app, pretending that the app signed something when it didn't. It's one thing to make a change in the database's state, it's another to claim that an app or user asked you to do it.

• Quantum Resistance. When quantum computers are capable of breaking elliptic curve cryptography, etc. then anyone with sufficient resources can impersonate an app's requests. Well, this is true, but when the time comes we can switch to using SPHINCS+ which is also based on hashes, or perhaps lattice-based cryptography. It's true that we'll need time for apps to switch, but unless you're building a distributed protocol, you can force their hand by simply changing the platform.

• Old versions of PHP don't support public key cryptography out of the box, and we don't want our users to have to install custom PHP extensions. I could understand this rationale specifically for Wordpress, Discourse, and other open source platforms that are meant to be self-hosted. But now all the languages do support it out of the box. So what's the problem? Plus, most APIs are run by centralized companies, so this was neevr an issue.

In short, I do not understand why APIs use shared secrets instead of public keys for their API clients.

Answer 1

To go along with what others have said, I also don't think this is quite right:

if the platform's database is compromised, all the secrets are leaked

The server should be storing a hash of the API key, not the API key itself, like we do with passwords. That way, a leak of the database would not allow an attacker to make fraudulent API requests (and since the API keys are long and random, any decent hashing scheme would not be practical to brute force either).

All services that I recall coming across that provide API keys have only allowed you to view the key when you initially generate it, not afterwards, which would support the idea that they are being stored in an irretrievable format.

Answer 2

I don't think it makes sense to call these kinds of tokens "symmetric keys", because they are not encryption keys that are used as input to a symmetric cipher. They are bearer tokens that are protected in transit by an entirely separate encryption system (usually TLS). They are often referred to as "API keys", but that's unrelated to how the word "key" is used in the context of cryptography.

---

I think the answer to your question is that asymmetric cryptography is more complicated than using bearer tokens, without providing any meaningful advantages.

The only arguments you suggest for using asymmetric encryption are these:

One is that if the platform's database is compromised, all the secrets are leaked.

In general, the data that is protected by an authentication/authorization system is much more valuable than the authentication token itself. (The exception would be something like a password that might be reused across multiple sites, but that's not an issue with randomly-generated API keys.)

If an attacker can fully compromise the platform's database, they don't need to authenticate. they can just retrieve (or alter) the sensitive data directly. It doesn't matter that the tokens were leaked.

On the other hand, if an individual user's token is compromised, you can always revoke their token, just as easily as you could revoke their public key if their private key was somehow compromised.

Second is that the platform may impersonate the app, pretending that the app signed something when it didn't. It's one thing to make a change in the database's state, it's another to claim that an app or user asked you to do it.

OK, but's not that simple. You're talking about enabling the system to cryptographically prove that its current state is a result of only authorized user actions. That means requests have to not only be digitally signed, but also stored in a permanent log somewhere. And you have to be able to prove that the current state results from reconstructing all of those actions. That adds a significant amount of implementation complexity.

There are systems that work that way -- notably, Git (with signed commits) and various blockchains -- but for most applications, there is no significant user demand for such a level of verifiability. In most cases, if a user thinks your system is deliberately tampering with their data, they won't waste time asking for cryptographic validation. They'll just stop using it.

(And of course, even a digital chain of trust doesn't truly prove that the authorized user was the one who performed the request, because their key could have been compromised.)

Answer 3

If I'm talking to an API endpoint that uses a symmetric key, I can easily invoke that using cURL on the command line or fetch in Javascript. A junior developer could integrate with the API in just a few minutes.

If I want to sign my requests, I need to use a third-party library. This greatly increases developer time and integration complexity. The service would incur more in support costs to assist developers with integration problems, and it would be more work to find and provide code examples for signing libraries in all of the different languages that developers may wish to use.

In my own personal experience, the quality of documentation for libraries accessible by simple HTTPS calls has been much higher than the documentation for individual libraries that have been made available by the service for a variety of different languages. Only larger companies with a big budget can afford to do a great job at developing, documenting, maintaining and supporting a dozen different libraries.

Answer 4

As Colonel Thirty Two said:

"What threat model" you are trying to protect against?

Threat Models

A breach against the client would provide the attacker access to the private key as easily as to the token/symmetric key.

Assuming the use of SSL for TLS - You are already using private key cryptograph for TLS, so with some hand waving you basically have the same threat model at the transport layer either way.

When we consider the server side: If an attacker has write access (to the database) they can simply substitute their own credentials so there, isn't a big win in this case.

I will concede that if a hacker only has read-only access either to the database or some way to sniff the key/token on the server side then you are correct they can mount an attack that otherwise wouldn't be possible - if the database only stored a public key.

Real Answer

The real answer to your question is "it's not worth it". Using an asymmetric key would mean:

Key Transfer Protocol

The server needs a process for you to authenticate and provide your public key.

Where as most API tokens are usually generated after some contract negotiation, by someone at the servers company and simply transferred out of band.

How is the Asymmetric key used?

You could develop your own transport protocol ... why ?

You could use the asymmetric key to generate a session token. However the server needs to retain that state / distribute it to multiple nodes - and you now run the risk of a session take-over given the theoretical "read-only attacker".

You could sign every payload - but that would likely be more expensive (on both the client and server sides) than a simple token check.

Don't Reinvent the Wheel

Finally, I will point out that SSL already supports client certificates, hence that would be an option, if the industry saw this attack as a significant enough problem.

Answer 5

In my experience there’s a better solution. If you look at most apis they just include the api key in the header. I believe this is the root of your concern is that it’s not hidden and there is no way to authenticate the content of the message.

Pub/Priv keys are not the best tool to address this. Check out AWS api authentication methodology. The api key is used to generate a signature of the request and that is recalculated on the server. This is a much more secure way to prove but authenticity of the sender but the content of the message as well.

Answer 6

The TLS infrastructure is already there to secure the connection between the endpoints, so just slipping a shared secret or bearer token inside of it for authentication (i.e. you are who you say you are) is the path of least resistance.

When it comes to asymmetric key logins (i.e. be your own certificate authority!), browsers have supported cert-based crypto+auth for a long time, but it's buried in the settings page, and the location differs from browser to browser.

For tech people who use ssh on a daily basis, a cert-based-auth web (i.e. authenticating clients via keypairs, in addition to servers) would have its merits. Though for the average user, openssh-style trust-on-first-use (TOFU) for server connections & keypair managment for clients is probably going to be more than they can competently handle.

Personally speaking, I'd be willing to roll the dice on using ssh-style crypto+auth for the web. TLS is centralized, over-complicated, and makes it very easy for powerful entities to MITM a conversation[0][1].

[0] https://en.wikipedia.org/wiki/Certificate_authority#CA_compromise

[1] https://arstechnica.com/information-technology/2013/01/turkish-government-agency-spoofed-google-certificate-accidentally/