Hexadecimal is traditional -- by this, I mean that there first were command-line tools that used hexadecimal for output, then other people using the hash functions found it fit to stick to hexadecimal, if only to be able to compare their values with the output of the aforementioned tools. That's how traditions get established: a more-or-less random choice at the start, then the need for interoperability and backward compatibility kicks in.
In the case of hexadecimal in cryptographic algorithms, one can probably trace it to the use of C language for reference implementations. Most algorithms are described with a specification (mathematical description, usually typeset in LaTeX), and a reference implementation that produces basic test vectors. For better or worse, the reference implementation is usually in C (or sometimes C++). In C, there is no standard facility for Base64 encoding (some programming platforms offer that, or external libraries, but it is not standard); but hexadecimal is easily obtained with a simple printf()
with a "%08x
" format string. As a very classic example, consider the MD5 specification (RFC 1321), which contains a reference implementation that does hexadecimal output.
The tradition is well entrenched; for the SHA-3 competition, NIST actually asked for reference implementations in C, and known-answer tests with a fully-specified text format that was hexadecimal throughout.
It must also be said that hexadecimal is convenient for debugging: the human developer can easily observe hexadecimal output and map these to individual bits, by doing the simple conversion in his head. Base64 is not as simple, because it entails 64 glyphs instead of 16, including some which are prone to induce visual confusion (1 vs I vs l, 0 vs O...). Also, many algorithm internally use 32-bit or 64-bit words, that map well to CPU registers; 32 and 64 are multiples of 4 but not of 6, so Base64 encoding again implies some non-trivial splitting.