# How does converting character set affect hash entropy?

Does mapping a large character-set hash onto a small character-set hash lower entropy for a fixed size substring?

I'm writing a python app that involves generating passwords for websites. During the application I want to convert hexidecimal hashes to alphanumeric strings, and take the first twenty characters of the output. I want to convert to alphanumeric, because most websites at least support the upper-lower alphanumeric character set.

I surjectively map the larger character set of hexadecimal bytes $\{00...ff\}$ onto the smaller set of upper and lowercase alphanumerics $\{a...z, A...Z, 0...9\}$.

The mapping is done by recycling the 62 alphanumeric elements cyclically to correspond to each element of the 256 bytes

$$00 \rightarrow a$$ $$01 \rightarrow b$$ $$\vdots$$ $$0e \rightarrow 9$$ $$0f \rightarrow a$$ $$\vdots$$

For example, if the world's most secure password - "password" - is sha512 digested as hexadecimal it is unsurprisingly - $128 \times log_2(16) = 512$ bits.

$$b109f3bbbc244eb82441917ed06d618\\ b9008dd09b3befd1b5e07394c706a8b\\ b980b1d7785e5976ec049b46df5f1326\\ af5a2ea6d103fd07c95385ffab0cacbc\\ 86\\$$

Since each pair of hex-digits is mapped onto one alphanumeric digit, I assume the entropy of the resultant alphanumeric string is $64 \times log_2(62) = 382.5459$ bits.

$$1j5bcKq8KdvcwVJpuiJj3efBGh5oYSp\\9 e1D6GB4YeFiLHtMZCUQxdfhpvjhVmWck\\$$

The new string has half the length but ~75% of the entropy of the original string, so there is more per-character entropy in the alphanumeric projection, making

$$1j5bcKq8Kd$$

$$b109f3bbbc$$
Likely $0e\rightarrow 9$ and $0f\rightarrow a$ is wrong, I guess $3d\rightarrow 9$ and $3e\rightarrow a$ is meant. The result of SHA-512 has 512 bits, but not 512 bit of entropy when the input is a low-entropy password. The transformation applied to 512 bits with full entropy would not quite give $64\cdot log_2(62)$ bit of entropy, for symbol $9$ is less likely than symbol $a$ is; I get $380.8\dots$ bit using the classic formula. The assumption that an alphanumeric password is valid fails for my email provider. – fgrieu Feb 19 '14 at 8:29