They are asking me to deliver encrypted text in ISO/IEC 8859-1. I have always encoded it to hexadecimal or base64. This text will be sent in a file with plaintext and encrypted text. In my opinion it won't work because of the non-visible characters.

I also wonder if there are characters that exist in hexadecimal and not in ISO/IEC 8815-1 (Western-Latin encoding). Am I right in assuming so?

  • $\begingroup$ Welcome to Cryptography. This is your answer Why do we use Base64? from Stack Overflow, and off-topic here. $\endgroup$ – kelalaka Dec 13 '20 at 20:31
  • 1
    $\begingroup$ ISO 8815 deals with Aircraft Electrical cables and cable harnesses. That's squarely off-topic :-) Base64 is a Binary-to-text encoding. One is required for the ciphertext of common algorithms (like AES-GCM) if the restriction of ISO/IEC 8859-1 to like 191 byte values out of 256 is enforced by whatever the file's data goes thru. Base64 can be used. $\endgroup$ – fgrieu Dec 14 '20 at 6:10
  • $\begingroup$ Sure enough, I put it wrong, it seems like I'm flying .... :) Sorry, because my English is a bit bad, I don't quite understand the phrase: "if the restriction of ISO / IEC 8859-1 to like 191 byte values ​​out of 256 is enforced by whatever the file's data goes thru. Base64 can be used" The curious case is that within that file there are two lines in plain text (header and footer) and the rest encrypted. $\endgroup$ – Felipe Rodriguez Fonte Dec 14 '20 at 7:26

Binary ciphertext may contain any byte value. However, character encoding's such as ISO/IEC 8859-1 don't allow all bytes to be used. Furthermore, some of the characters defined (in the range 00 to 1F) are so called control characters that are not "printable", i.e. they don't show up as characters in text. Because of this you need to encode ciphertext if you need to store or transmit it as text.

Hexadecimals uses the characters 0-9 and a-f and / or A-F. As such they can be used in any text document. Note that hexadecimals are not binary themselves, although they can be used to represent a byte with any value using precisely two hexadecimal digits 00 to FF. I think the main issue you are having is not understanding the difference between binary and the representation of the binary (bytes) using hexadecimals.

Base 64 uses the characters A-Z, a-z, 0-9 as well as / and + Finally it uses the padding character =. It also always fits in any text, UTF-9 or ISO/IEC 8859-1. It's no different than hexadecimals in that respect, although it is of course more space efficient but less "readable". It is less readable because it is harder to calculate the size and determine which bits are set from base 64 encoding compared to hexadecimals).

So the idea is to:

  1. perform character encoding on the plaintext such as UTF-8 to create a binary plaintext (this is only required if the plaintext is not binary already);
  2. encrypt the binary plaintext;
  3. encode the binary plaintext to "text" using hex or base 64 encoding.

During decryption this is performed in reverse: hex / base 64 decoding, decryption and then possibly character decoding to get back the original text.

In my opinions, encoding of ciphertext is performed much too often. Generally it is better left as binary. For instance, you can store it as file, VARBINARY in SQL databases or send it as binary over HTTP without major issues.

Both hex and base 64 encoded ciphertext are not very well suited to be directly embedded in human readable text Some kind of way is required to distinguish it from "normal" text after all. PEM encoding uses header and footer lines for that reason.

There are even more formal ways of doing this: XML and - to a less extend - JSON. With XML it is possible to use XML-enc to encrypt specific parts of an XML document, which is then usually stored as UTF-8. For JSON the JWE standard can be used to accomplish the same thing.

  • $\begingroup$ I think the encoding of ciphertext is performed much too often is about being conservative? You may need to send the data that may require base64 like coding to be transferred. Of course, we are not talking about the new designers that always applies that when thye heard others do. Please use HTTPS :) $\endgroup$ – kelalaka Dec 14 '20 at 13:15
  • $\begingroup$ I'm very afraid that it's more herd behavior than anything else :| $\endgroup$ – Maarten Bodewes Dec 14 '20 at 13:33
  • $\begingroup$ Thanks a lot!!! $\endgroup$ – Felipe Rodriguez Fonte Dec 14 '20 at 20:31

When using standard encryption algorithms like AES-GCM, ciphertext is bytes that are approximately uniformly distributed among the 256 possible byte values. Since ISO/IEC 8859-1 allows like 191-byte values out of 256, it's extremely unlikely that any practical ciphertext will entirely consist of these 191 bytes, as would be the case if some appropriate Binary-to-text encoding was used, including Base64. In the rest, I'll assume it is not used such encoding.

The curious case is that within that file there are two lines in plain text (header and footer) and the rest encrypted.

That does not make it impossible for decryption of such files to works. Many modern file transfer methods, including USB thumb drives, attachments to email or messaging, network shared volumes, cloud storage, UNIX pipes, windows pipes opened in binary mode… will happily transfer any (contiguous and single fork/stream) file with all its data (if not metadata like file name, date and security attributes) unchanged. Thus the problem of allowing decryption boils down to locating the ciphertext in the file.

That can be dealt with with conventions. One possibility is that ciphertext starts after a header of fixed size in bytes that the decryption skips. Another is that it starts after the first occurrence of a fixed text sequence, e.g. <-- ciphertext on next line --> followed by a single LF character. The end of ciphertext can be dealt with by embedding the ciphertext length at the start of ciphertext. That can work if the file is correctly prepared, e.g. by a correctly written program; or by manually concatenating a header file, the ciphertext, and the footer file using bash on a Unix system; or using copy /b of cmd.exe on Windows.

Things that go wrong when that's attempted include: the file is opened and mistakenly saved with a text editor (perhaps to remove the header), and that particular text editor ruins the ciphertext (many do); the sender used CR LF as end-of-line and the decryption is unaware of that; the file is piped to a console display emulating some glass terminal (e.g. a VT100 itself emulating an ASR-33), which emits silly noises, locks, or act strangely because it entered an obscure mode.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.