Let's say I have an API that provides an encryption wrapper around another API. I give the consuming programmer the option of either passing a string or byte array as an encryption key.
Encrypting with a proper random key and doing so with a password are different enough that I would opine that an API should strive to offer them as different operations, not as two modes of one operation. Moreover, the routine to encrypt with a password would want to bottom out to the key-based encryption, so code factoring calls anyway for there be a function for the latter.
How to implement this depends very much on your language's type system as well. For example, the Rust programming language—which is noted for having a well-designed static type system—the types you'd likely choose for a binary key and for a password would not be the same:
- For a binary key you'd probably use a type like
&[u8] (an unsigned byte slice (pointer + length into some array)) or
&[u8; 16] (reference to a whole array of 16 unsigned bytes).
- For a password you'd probably use
&str, a string slice—pointer + length at a span of UTF-8 text).
By using a statically typed language that distinguishes at compilation time between binary data and text, such an API would therefore be much more likely to catch client errors at compilation time when a caller tries to pass a textual password to the raw encryption function or binary data to the password-based one. More generally, you've made it so that the path of least resistance is for the programmer to do the right thing.
In addition, the password-based API function shouldn't be implemented by simply hashing the password with one of the SHAs; it should use a dedicated password-based key derivation function like Argon2.