XOR of 16-bytes variables gives a result of size not equal to 16

Question

I'm trying to write a XOR code for implementing CBC mode of operation from scratch but i seem to have some problem.The function takes two byte variables of 16bytes and the expected output should be 16bytes which we get it from the first line in main but when that string is converted to bytes the size increases. Feels like I'm misunderstanding something

from binascii import unhexlify
import os

def xor(string1,string2):
    list1=[]
    for i in range(len(string1)):
        list1.append(chr(string1[i]^string2[i]))
    return (("".join(list1)))

if __name__ == "__main__":
    print(len((xor(os.urandom(16),os.urandom(16))))) #always comes out 16 #type string
    print(len((xor(os.urandom(16),os.urandom(16))).encode("utf-8"))) #never equal to 16 #type byte

Answer 1

In implementing crypto, it is essential to distinguish strings (of characters) from byte vectors, also known as bytestrings or byte arrays.

In most computer languages, strings store characters, not bytes, and are not adequate for (un-encoded) intermediary values, ciphertext, keys, Initialization Vectors.. even when and if adequate for plaintext. In C, characters can be signed, and 0 ends a string. In Java, strings contain 2-byte characters. In many modern languages, strings contain abstract Unicode points, or a subset of that, or their representation as bytes per UTF-8 (1 to 4 bytes per character).

The first computational step of encryption of a string is to convert it to byte vector. In modern Python practice, that can be

myString = 'Per Pythagoras: x²+y²=z²'
myByteVector = bytes(myString, 'utf-8')

which here turns a 24-character string into a 27-byte vector. It happens that with UTF-8 encoding, what an American considers a normal character takes one byte, except when it doesn’t (that’s two examples since the previous comma!).

Then byte vectors can be processed, XORed.. (typically, much more naturally than strings) and won't change length unexpectedly.

On decryption, the byte result needs to be changed back¹ to characters, perhaps as a string. In modern Python, the idiom can be

myString = str(myByteVector, 'utf-8')  # can cause exception!

Note: when and if a ciphertext, key, Initialization Vector.. needs to be represented as string, it should be encoded per some binary-to-text encoding, such as the popular Base64.

---

¹ Beware that's not always possible, e.g. when the ciphertext was altered or the deciphering key is wrong. In such case, the best outcome is an exception; others include garbage text, a beep when that's displayed, a devastating security vulnerability, execution of the HCF instruction, torn out paper on a Teletype.