4
$\begingroup$

Given a cryptographically secure series of octets, it is often useful to display this information or store it in a database in a textual format such as hexadecimal.

Why would it be incorrect to take this hexadecimal and encode it with UTF-8 to derive a symmetric key rather than converting from hexadecimal back to the original byte array?

My thought is that it reduces the key space and presents a bias toward where hexadecimal characters fall in the UTF-8 encoding scheme. Perhaps someone can clarify this for me.

$\endgroup$
4
  • 1
    $\begingroup$ Not all random byte strings are valid UTF-8, for starters. Also, I think you may be unclear on exactly what the difference is between an encoding versus the underlying value being encoded. $\endgroup$ Mar 31, 2016 at 22:33
  • 1
    $\begingroup$ @StephenTouset I want to be clear - I understand the difference but I am having a hard time explaining to a coworker why it's not a good idea in more technical or mathematical terms. Their argument is that since a single byte is 0 - 255 and a single character is 0 - F that they actually double the key strength by encoding it since each character would get represented by two bytes; obviously that's wrong. I argued that the entropy of the source remains the same and that because we would need to then truncate the string that the overall entropy of the key is reduced by half or more. $\endgroup$ Mar 31, 2016 at 22:42
  • 6
    $\begingroup$ @MichaelJ.Gray your coworker is an idiot $\endgroup$ Mar 31, 2016 at 23:26
  • 2
    $\begingroup$ I'm often telling people on StackOverflow that hexadecimals is just the human representation of the bytes. What you want to use is the actual value of the bytes. You can also put it in extremis and propose to use the binary (base 2) representation as key. Unless they're extremely daft they probably see the issue against using "00010010110001001000010100100010001001001010 (etc.)..." in ASCII. $\endgroup$
    – Maarten Bodewes
    Apr 1, 2016 at 8:51

1 Answer 1

9
$\begingroup$

The strength of a symmetric key is determined by the amount of entropy. In that sense your scheme would not increase or decrease the security by much. However, you would be specifying a higher key size than actually used. For instance your AES-256 bit key would have the same entropy as an AES-128 bit one. You'd use a slightly different key schedule and number of rounds as well of course.

Besides that you may run into problems when somebody decides to use a different encoding (hex encoding in UTF-8 is the same as just using ASCII though) or a byte order mark. It could also be possible to mix up uppercase and lowercase characters.

In the end you're only fooling yourself and possibly others if you'd use this kind of scheme. It is very bad practice. If you have to store the key as hexadecimals then pretty please with sugar on top, convert it back to the original form.

In general though you should never ever have to store a key in a string in the first place. Note that strings may be very hard to delete from memory. Instead store them in a key store such as PKCS#12 or preferably in a hardware device.

$\endgroup$
3
  • 1
    $\begingroup$ I appreciate the reply and the explanation. I know I left my context agnostic to the algorithm we're using, but it's HMAC-SHA512 and your point is even more valid in this case because of the way HMAC is specified to handle keys longer than the internal block size of the hash. If we used a 512 bit key it would end up as a 1024 bit ASCII block which would get hashed internally and then used as the key. That would make it difficult for other software to integrate since it's not a standard practice. $\endgroup$ Mar 31, 2016 at 23:43
  • $\begingroup$ @MichaelJ.Gray HMAC is pretty resistant against abuse, even though it does recommend a key input the same size as the hash. I'd be more worried about the other software to use a different case for hexadecimals. Although there is an RFC, the use of uppercase or lowercase isn't really well standardized in practice. $\endgroup$
    – Maarten Bodewes
    Apr 1, 2016 at 22:39
  • $\begingroup$ Just to clarify, we're not using a key that is a hexadecimal string. The hexadecimal conversion is only because the value is stored in a table that doesn't have a BINARY data type for the column it needs to be in. The idea is to encode the hexadecimal (using the normal hexadecimal conversion, not UTF-8) into bytes to be fed into HMAC. I agree with you. $\endgroup$ Apr 4, 2016 at 20:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.