Binary ciphertext may contain any byte value. However, character encoding's such as ISO/IEC 8859-1 don't allow all bytes to be used. Furthermore, some of the characters defined (in the range
1F) are so called control characters that are not "printable", i.e. they don't show up as characters in text. Because of this you need to encode ciphertext if you need to store or transmit it as text.
Hexadecimals uses the characters
a-f and / or
A-F. As such they can be used in any text document. Note that hexadecimals are not binary themselves, although they can be used to represent a byte with any value using precisely two hexadecimal digits
FF. I think the main issue you are having is not understanding the difference between binary and the representation of the binary (bytes) using hexadecimals.
Base 64 uses the characters
0-9 as well as
+ Finally it uses the padding character
=. It also always fits in any text, UTF-9 or ISO/IEC 8859-1. It's no different than hexadecimals in that respect, although it is of course more space efficient but less "readable". It is less readable because it is harder to calculate the size and determine which bits are set from base 64 encoding compared to hexadecimals).
So the idea is to:
- perform character encoding on the plaintext such as UTF-8 to create a binary plaintext (this is only required if the plaintext is not binary already);
- encrypt the binary plaintext;
- encode the binary plaintext to "text" using hex or base 64 encoding.
During decryption this is performed in reverse: hex / base 64 decoding, decryption and then possibly character decoding to get back the original text.
In my opinions, encoding of ciphertext is performed much too often. Generally it is better left as binary. For instance, you can store it as file, VARBINARY in SQL databases or send it as binary over HTTP without major issues.
Both hex and base 64 encoded ciphertext are not very well suited to be directly embedded in human readable text Some kind of way is required to distinguish it from "normal" text after all. PEM encoding uses header and footer lines for that reason.
There are even more formal ways of doing this: XML and - to a less extend - JSON. With XML it is possible to use XML-enc to encrypt specific parts of an XML document, which is then usually stored as UTF-8. For JSON the JWE standard can be used to accomplish the same thing.