# CTR HMAC SHA256 implementation

After trying to invent my own AES mode, I decided to just implement something that's known to work, namely CTR with HMAC SHA256.

1. From what I understand, I can use the IV as a counter, and the IV only has to be unique for each key (generating unique IVs isn't a problem). Correct?

2. Is it a problem for HMAC SHA256 if I construct a SHA block using a 32 byte key, 16 byte counter and 16 zero bytes for padding, or is it better to generate a 64 byte key so it fits in one SHA block and make another block for the counter and pad that block with zero bytes?

3. If I want to add a salt to anything, I assume the salt is stored unencrypted. Correct?

Edit for clearing up some things:

Using the IV as a counter implies that it changes for each encrypted block. In this case I'll use a simple time stamp directly from the OS that's 12 bytes large (resolution is in ticks, which are 1/50th of a second, software waits one tick before generating an IV, system is slow so not a problem). The remaining four bytes are used as a block counter (simply starts at zero).

The intended HMAC setup is:

The first SHA256 message is made of the first HMAC key, the IV, padding and the cipher text. The first SHA256 block will be the first HMAC key (32 bytes), then the 16 byte IV, then 16 bytes of zero padding. The remaining blocks for the message are cipher text blocks, with normal SHA padding after the last cipher text block (normal SHA operation). The resulting hash is the first HMAC hash.

The second SHA256 message is the second HMAC key (32 bytes) and then the first HMAC hash (32 bytes), with the normal SHA padding at the end. This is the authentication tag.

Second edit:

After searching the web some more, I found that I made some assumptions about the way the HMAC keys are used. I've used some test vectors from NIST to verify that I'm doing things properly, and that's now the case.

Proper HMAC SHA256 setup:

Take first 32 byte HMAC key, append 32 zero bytes, XOR IPAD over the resulting 64 byte block, append cipher text, hash.

Take second 32 byte HMAC key, append 32 zero bytes, XOR OPAD over the resulting 64 byte block, append first hash, hash.

I assume you can prepend the nonce to the cipher text so it gets hashed as well. Correct?

• What would you be adding a salt to? This sounds worryingly offhand, as if randomly tossing in a salt here and there might improve security. – Stephen Touset Mar 10 '15 at 2:43
• The salt is for generating keys from passwords. It's actually separate from this. – Thorham Mar 10 '15 at 2:47
• Great, just checking. – Stephen Touset Mar 10 '15 at 2:52

From what I understand, I can use the IV as a counter, and the IV only has to be unique for each key (generating unique IVs isn't a problem). Correct?

Not exactly. The IV needs to be unique for each block per key. You can use a message counter for the Nonce component of the IV, then the block counter component is incremented from 0 in CTR.

IV = [96-bit Message NONCE][32-bit Block COUNTER]


Since the Nonce does not to be random or unpredictable, a message counter is appropriate. I use a combination of message counter, time code, and random number, all 32-bits, which helps against accidental nonce reuse if the counter gets maliciously reset. The key needs to be changed before the message counter overflows.

IV = [32-bit RANDOM][32-bit Timecode][32-bit Message CTR][32-bit Block CTR]


There are other methods that can be used. You may not want timecode information to be sent with a message, or have an attacker know how many messages you have sent. You can even encrypt the nonce with another key before using it in CTR mode, but you will need a custom setup for that because it can be done VERY wrong and break security.

Is it a problem for HMAC SHA256 if I construct a SHA block using a 32 byte key, 16 byte counter and 16 zero bytes for padding, or is it better to generate a 64 byte key so it fits in one SHA block and make another block for the counter and pad that block with zero bytes?

That will change how HMAC operates, which is to only have the key as the key input. HMAC per spec will only hash the key input if it is larger than the blocksize, so you can input 512-bits of "key" data, although it is not necessary to include a counter as part of the input. Appending the nonce to the key should not have any negative effect on security, it may even strengthen it, but I will not say that with any certainty. It is probably better to prepend the nonce to the message input if you want it to be authenticated, rather than adding it to the key. Including the counter with the key also eliminates a specific kind of performance optimization you can use.

You may want to consider SHA512/256 or SHA3-256 instead. If you have 64-bit code and CPU available it is faster, and should be more secure against current and unknown attacks. MAC using SHA3 is different, it prepends the message with the key.

If I want to add a salt to anything, I assume the salt is stored unencrypted. Correct?

A salt is a non-secret piece of cryptographic information, so it does not need to be encrypted. You may want to add some kind of redundancy check or rebuild parity (such as Reed-Solomon) to be associated with it, if you feel there is a chance it can be altered accidentally. Keep a backup somewhere, a password without the salt will mean a lost key.

Finally, make sure you are using different keys for your cipher and your MAC. The reason is that an attack on one of them can break both. While that may be unlikely, it is still a good idea to use separate keys, and is FIPS compliant.

• Hmm, I think I've been a little unclear about the IV and the HMAC setup. See edit of original post. – Thorham Mar 10 '15 at 2:47
• @JohnSmith no it was clear enough for HMAC. Using an OS time code is not a good idea for nonce generation – Richie Frame Mar 10 '15 at 4:45
• Yeah, you're absolutely right. I made some assumptions about the HMAC keys. Why isn't an OS time stamp good for a nonce? – Thorham Mar 10 '15 at 12:06
• @JohnSmith malicious modification of the time is quite easy. If it is from boot it resets at reboot, if it is from a certain time, the current time can be changed – Richie Frame Mar 10 '15 at 19:26
• That's true. I was going to tell the user to make sure the time and date are correct. I don't suppose it's an option to let the user supply the time stamp? – Thorham Mar 10 '15 at 20:10

I decided to just implement something that's known to work, namely CTR with HMAC SHA256.

So you are implementing your own mode after all? Don't. Combining MAC and encryption is actually tricky. You should use a standard authenticated encryption mode, such as GCM (which is CTR plus MAC done right).

From what I understand, I can use the IV as a counter, and the IV only has to be unique for each key

For CTR, the IV (more properly termed initial counter) needs to be unique for each block, for a given key. For example, if you start encrypting a 3-block message with the counter value 0, this also uses up the counter values 1 and 2, so the next message must start with counter value 3. The initial counter doesn't need to be random (but that's usually a good way of making it unique). It's ok to use the same counter with different keys.

Is it a problem for HMAC SHA256 if I construct a SHA block using a 32 byte key, 16 byte counter and 16 zero bytes for padding, or is it better to generate a 64 byte key

I don't understand this. HMAC-SHA256 uses a 32-byte key. The size and structure of the message is unrelated to the size of the key.

If I want to add a salt to anything, I assume the salt is stored unencrypted.

Yes, that's the definition of a salt: something that needs to be unique (within a certain domain) and that doesn't need to be secret.

• I'm not using my own mode. I'm using AES CTR as the cipher and HMAC SHA256 as the authentication mode.As for the IV/counter, I understand that the counter can't stay the same for each block, it's a counter after all. – Thorham Mar 9 '15 at 22:09
• @JohnSmith But combining them makes a new mode, AES-CTR+HMAC-SHA256. – Gilles 'SO- stop being evil' Mar 9 '15 at 22:11
• Why? How is picking an encryption mode and an authentication mode creating your own mode? – Thorham Mar 9 '15 at 22:13
• @JohnSmith Because combining them is tricky (see the threads I link to). “Creating your own” is bad because you're using primitives whose behavior is not well-known, so it's delicate at best to use them correctly. In the case of HMAC+CTR, it's known that it's a tricky combination, and neither encrypt-then-mac nor mac-then-encrypt can be used without being very very careful. Using a known authenticated encryption mode frees you of this concern. – Gilles 'SO- stop being evil' Mar 9 '15 at 22:18
• I only included both pads in the text to show that I'm not forgetting them. Everything is self written in assembly language based on existing C code (AES), pseudo code (SHA256) and explanations (HMAC). – Thorham Mar 9 '15 at 22:55

I was doing something like this, and discovered just how careful you need to be. I wanted to do password encryption using widely available primitives so that I could do re-keying within MySQL easily. I was already doing AES256 CTR and following the prescribed usage to the letter.

fileKey = random256Bits()
ivPerWrite = random256Bits()
//version 1 of the file
...
//version 2 of the file
ivPerWrite2 = random256Bits()


The block mode chosen matters in a deep way. You need to be careful about thinking of this as mostly equivalent ciphers (that you can negotiate with your attacker in some cases!). I picked CTR because I need to seek through the cipher and pull ranges of plaintext out of the ciphertext. You can have the same key for multiple versions of the same file, but you cannot reuse the IV when you write multiple versions of it. I was careful to ensure that every time I encrypted, I did so with a new IV. The short answer to why this is ... is that the last step of CTR is xor. CTR is a one-time pad with a pseudo-random keystream. Trivial pen and paper attacks are doable if the key stream repeats. It will repeat if IV+blockNumber is used under the same key for any block of ciphertext. So I got encrypting the file right. But...

Then I needed to encrypt the keys using a password somehow. You would think this would be hard to get wrong. So if you have a randomly generated key, and you xor it with pseudo-random bits (a sha256 hash of a password), then what could go wrong?. It was encrypting a permission between a user and an object. So I salted the hash with the user id to "diversify" the keys. I hadn't included in the hash the id of the object as well because I didn't have an id yet. But I only realized a bit later that this is isomorphic to the first block of a CTR. So "salting" isn't the issue, because the attack is not going to involve brute-force tables per user. The issue is (key,iv) repeating, just like with a one-time-pad. If you encrypt the same key with the same derived key, then the derived key can cancel, allowing you to perform algebraic attacks. Trent handles alice and bob's encrypted data. Trent generates a and b by hashing its secret with the username. Trent created all random keys x and y. But one day, bob gets his hands on the encrypted data:

alice granted x    (a+x)
alice granted y    (b+y)
bob granted x      (b+x)
bob forges a grant for y:   (a+x+a+y+b+x) = (b+y)


It is not leaking any keys or cipher texts yet, but it is allowing forgeries (that an attacker would need to get to Trent's database somehow). Because neither alice nor bob know any of the keys x or y (they belong to Trent), they can't cancel out x or y to get b or a. So the sha hash that included the password that went into a and b (along with the user id), should just be replaced with a completely random IV per encrypted key. So, make it never repeat so that you can't cancel anything:

(a + x)
(a' + y)
(b + x)
((a + x) + (a' + y)


Bob can't get cancellation, because every a and b are now unique, because they are generated like: sha256(pass+x.iv)

And note that even fully secure AES256 CTR doesn't prevent forgeries (same attack!) because the last step of any CTR is xor.

(grumpyCat.jpg + happyCat.jpg)
(a + happyCat)
alice forges (a + grumpyCat.jpg) from the ciphertext.


When you use a hashed password to encrypt a file, xor with the hashed password doesn't pass the secret through the hash. It effectively does an HMAC on an iv. To also authenticate that the encryption was done by the system, a sha256(password+encryptedKey) should be done as well.

Ok, then use a mac that can be created and verified by Trent so that Trent can not be fooled by new entries put into the database:

encKey = sha256(trentKey+permission.iv) ^ fileKey
authEncKey = sha256(trentKey+encKey)


This effectively produces a signature that Trent made and Trent can verify. It is separate from Trent's encrypted copy of the key that he can decrypt when he needs to.

So, if you look at how CTR works, it appears that a hash can be substituted where the cipher goes, because it's only used in one direction. But beware! It doesn't authenticate anything unless the data being encrypted and the secret pass through it. If you "salt" the hash (or equivalently put an IV into a cipher) and xor it with small random data like a key, an IV that never repeats should go into it.

def CTR(key, iv, totalBlocks, inBlocks, outBlocks, oneWayFunc):
for i in range(0, totalBlocks):
outBlocks[i] = inBlocks[i] ^ oneWayFunc(key,iv+i)


That oneWayFunc is usually a cipher. But CTR mode does not take advantage of it being reversible. It might as well be a hash. Notably, the data itself does not pass through this function. That means that being encrypted under this key in no ways implies that it was actually performed by Trent. That will need to be a separate step.