What is the need for further encoding the hash value?
Representing the hash as a string of characters, without increasing the size too much. This is known as Binary-to-text encoding. It is commonly used for cryptographic data (hashes, ciphertexts..), because that can contain arbitrary sequences of bits (or arbitrary sequences of arbitrary bytes), and some data communication, storage or display means can only handle characters in some alphabet.
This is not to be confused with encoding characters into bytes, as needed to hash or encipher text.
I will make the explanations and examples with the MD5 hash of a 26-ASCII-character message given in example in RFC 1321 appendix A.5:
MD5 ("abcdefghijklmnopqrstuvwxyz") = c3fcd3d76192e4007dfb496cca67e13b
The hash is not
c3fcd3d76192e4007dfb496cca67e13b. It really is a 128-bit sequence (or bitstring) which representation with
1) for a bit clear (resp. set) is:
The 32-character sequence
c3fcd3d76192e4007dfb496cca67e13b is a representation of the hash per lowercase big-endian hexadecimal. That was obtained from a representation of that hash as 16 bytes (or equivalently 16 octets) per this code C code (from the RFC):
/* Prints a message digest in hexadecimal.
static void MDPrint (digest)
unsigned char digest;
unsigned int i;
for (i = 0; i < 16; i++)
printf ("%02x", digest[i]);
Note: In the above,
unsigned char is a byte representing an integer in range [0…255] that encodes 8 bits of the hash. Conversion to characters is in the
printf function, with
"%02x" specifying at least 2 lowercase hexadecimal characters.
Note: Each group of 4 bytes is the little-endian representation of a computer word, but that's a consideration internal to MD5.
Beside the aforementioned representations as 16 bytes, and 32 lowercase (big-endian) hexadecimal characters, there are other common representations, including
C3FCD3D76192E4007DFB496CCA67E13B : 32-characters, uppercase (big-endian) hexadecimal
w/zT12GS5AB9+0lsymfhOw== : 24 characters, Base64 per RFC 4648
w_zT12GS5AB9-0lsymfhOw== : 24 characters, Base64 with Filename Safe Alphabet
w/zT12GS5AB9+0lsymfhOw : 22 characters, Base64 with pad suppressed
If is possible to convert from a text (equivalently, character) encoding back to the hash. The same hexadecimal decoding method can handle uppercase and lowercase. The same Base64 decoding method can handle common alphabets and make pad optional.
There are other, more seldom-used, sometime more compact Binary-to-text encodings. See this for why Base64 is most popular.
When we encode (a 16-byte MD5 hash) using the base64 scheme and produce 8 char long string, it will be 8 bytes.
No. Each Base64 character encodes 6 bits (except the last non-pad character which can encode 2, 4 or 6 bits; and final pad characters, if any). Therefore, per Base64 encoding, a 128-bit hash requires at least ⌈128/6⌉ = 22 characters, plus pad if any.
Depending on computer language and options, each character may in turn use several bytes. For example, each character in a C string is represented by 1 byte, in a Java
String 2 bytes. Also, there is size overhead for each object, which may not be negligible.
In a Python string, starting with version 3.3 or so of Python, each character in the above encodings by default uses 1 byte. In previous versions it used 2 or 4, depending on Python environment; refer to PEP 393.