1
$\begingroup$

The MD5 algorithm is designed to accept a message of any length and produce a 512-bit message digest. Because of the way it pads the original message, by adding a 1-bit then a number of 0-bits, the length of the message can be any number of bits. It is not limited to an integer number of bytes. But it can also be written to work only in bytes, so the padding stage would involve adding a 0x80 byte then a number of 0x00 bytes. The end result is the same, as long as the input data is an integer number of bytes.

However, my understanding is that data is almost always stored in bytes. This is especially the case for character strings, where each character is represented by at least 1 byte. For this use case, I believe a byte-oriented approach would work fine. But if I wanted to write an MD5 implementation that also accepts files, does the bit/byte distinction become important? Would I need to write the algorithm to deal specifically with bits, or would a byte-oriented implementation work just as well for strings (e.g. from STDIN) and files?

In other words, is data always stored as whole bytes? And if so, what applications are there in which one would need to write an MD5 algorithm that uses individual bits, rather than whole bytes?

$\endgroup$
2
  • $\begingroup$ Define "need to be written". I have a working implementation of SHA-1 (carried to SHA-256 when that came out) handling bit-sized data, to large size. I used it to make test vectors, and that was the one real application of it. But I can imagine an application where one wants a few variants of MD5, and obtain these by appending a suffix shorter than a byte to a bytestring then fed to MD5. That requires bit support, and for 15-byte messages makes a computational saving compared to appending a whole byte (1 hashed block rather than 2), plus avoids trivial collisions with regular MD5. $\endgroup$
    – fgrieu
    Sep 20, 2020 at 15:59
  • $\begingroup$ The context is that I'm writing an MD5 implementation from scratch, purely for learning. The application I'm targeting is basic console input. I would like my program to be able to accept a string from STDIN, so I could do something like md5 "some string" and have it return 5ac749fbeec93607fc28d666be85e73a. But I'm also considering functionality to accept a file. Therefore, could a byte-oriented approach work for both use cases, or would I need to write the algorithm to use individual bits? I've edited my question to make this clearer. $\endgroup$
    – AkThao
    Sep 20, 2020 at 17:09

1 Answer 1

2
$\begingroup$

If I wanted to write an MD5 implementation that also accepts files, does the bit/byte distinction become important?

No, a byte implementation is fine. In modern OSes, hashing from a file, pipe, string (whatever the encoding) is hashing bytes.

The only times I made and used an implementation of a hash with bit support was to make or check test vectors. But I can imagine a few use cases that would require bit support

  • hashing the expression as a minimum number of bits of an integer (per some specified endianness), for an hypothetical algorithm specifying that.
  • one wants a few variants of MD5, and obtains these by appending a suffix shorter than a byte to a bytestring fed to MD5. For 15-byte messages that makes a computational saving compared to appending a whole byte (one internal padded message block of 32 bytes rather than two), plus avoids trivial collisions with regular MD5.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.