2
$\begingroup$

Generating Authorization header with AWS signature v4 involves deriving signing key as follows:

https://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html

static byte[] HmacSHA256(String data, byte[] key) throws Exception {
    String algorithm="HmacSHA256";
    Mac mac = Mac.getInstance(algorithm);
    mac.init(new SecretKeySpec(key, algorithm));
    return mac.doFinal(data.getBytes("UTF-8"));
}

static byte[] getSignatureKey(String key, String dateStamp, String regionName, String serviceName) throws Exception {
    byte[] kSecret = ("AWS4" + key).getBytes("UTF-8");
    byte[] kDate = HmacSHA256(dateStamp, kSecret);
    byte[] kRegion = HmacSHA256(regionName, kDate);
    byte[] kService = HmacSHA256(serviceName, kRegion);
    byte[] kSigning = HmacSHA256("aws4_request", kService);
    return kSigning;
}
  1. What is the benefit of calculating HMAC as shown above, in multiple subsequent steps, instead of doing doing just one HmacSHA256 calculation over concatenated input (dateStamp, regionName and serviceName)?
  2. What is the benefit of the above approach instead of using standard HKDF?
$\endgroup$
3
$\begingroup$

The staggered scheme allows each step to be performed by a different entity.

  1. A central server stores the master key and does not communicate it to anyone else. This central server should have high availability and high security, but can have low throughput. The key can be stored in a hardware security module.
  2. The first step combines the key with a protocol version number in an unambiguous way ("AWS4" + key). This allows a protocol upgrade to use the same key, so that you don't have to distribute new keys at the same time as the protocol upgrade.
  3. The second step combines the key with the date (HMAC(dateStamp, "AWS4" + key); the protocol specifies a date stamp with a 1-day resolution). The key changes every day. Quicker changes require more throughput on the central server, whereas slower changes mean a larger impact in case something down the chain is compromised.
  4. Every day, each region server requests the day's region key kRegion = HMAC(regionName, HMAC(dateStamp, "AWS4" + key)). The central server can authenticate the region server to ensure that only a given region's server can obtain the region key.
  5. Each service that uses the key requests its own key kService = HMAC(kRegion, serviceName) from the region where it's running. The region server can authenticate the service to ensure that a service doesn't learn another service's key.
  6. The service combines its key with a purpose: kSigning = HMAC(kService, "aws4_request"). This allows the same master key to be used for different purposes if needs be.

The advantage of this staggering is that each participant only has access to its own keys. It's impossible to find the master key given kRegion, or to find the region key given kService, or to find the service key given kSigning. This minimizes the impact of a compromise: only participants down the chain are affected. For example, if a breach is discovered in the American datacenter on 2022-01-05 and the logs show that the breach occured on 2022-01-01 then only signatures made after 2022-01-01 in the American region are suspicious and may need to be revoked. Signatures made in Europe or before 2022-01-01 are still good.

Of course only large deployments will ever make use of this flexibility. In a majority of applications, the whole calculation is done in the same piece of software in a single function. But the protocol does scale to large deployments that use compartmentalization to mitigate the impact of breaches.

Combining the inputs all at once:

$\endgroup$
1
  • $\begingroup$ @automatictester Indeed, this was explained in one of the keynotes this year re:Invent. The main purpose is to delegate the authentication to individual regions /services. Each service would receive a derived key for the user once valid for the day, so the service itself doesn't get access to the secret key, yet the service woudn't need to contact the central IAM for each request to authenticate. $\endgroup$
    – gusto2
    Dec 26 '21 at 21:11
1
$\begingroup$

So the problem AWS is solving here is that they have a tuple of values, namely the service, the region, and the current date and they would like to derive a different secret key that depends on all of these values.

You could go ahead and do this with a regular PRF but then you run into a problem: Namely you don't want to have a situation where bad values of any of the tuple entries suddenly yield key collisions. A classic mistake would e.g. be to use straight concatenation and if they're not careful you end up with (to,day) and (tod,ay) yielding the same key. Obviously the strings in play here have more structure than the example, but the underlying worry should be clear.

Now, the most efficient way to resolve the above issue is to use a pairing function that guarantees that no two distinct input tuples yield the same output and thus the same input to the PRF. However, designing such functions is also mildly error-prone.

I guess it is because of this that AWS chose to use this convoluted chain of HMAC invocations. Now they don't have to worry about complicated pairing functions and all that is required is a bunch of HMAC evaluations which should be possible in almost all languages / environments. The fact that this then requires no less than HMAC operations is offset by the fact that all the inputs to this key derivation are somewhat static and you can cache the keys for about a day if HMAC operations cost considerable compute time for you.

$\endgroup$
3
  • $\begingroup$ Isn't calculating HMAC once over structured string like [dateStamp=...;regionName=...;serviceName=...] an easier solution to this problem? $\endgroup$ Jul 30 '21 at 6:26
  • 1
    $\begingroup$ @automatictester it's just another solution, a design decision. When the "structured" input may contain any client input (s3 key, db row id,.) then I would rather use nested hash than risking any potential conflict. $\endgroup$
    – gusto2
    Aug 27 '21 at 19:38
  • $\begingroup$ @automatictester Why do you think it would be easier? Compare the number of lines of code for the two solutions, and note how if the structured string is easy to construct, it means you're adding a large data formatting library as a dependency. $\endgroup$ Dec 24 '21 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.