We are building out data masking framework mainly to mask PIIs. Our scale is pretty large, and masking will be done at ingest time, so we want the masking to be done in a very performant manner. Some of the constraints we have are that we would like the masking to be deterministic and reversible. I have looked at AES encryption to encrypt PII, especially AES SIV, on my macbook, it takes around ~2 milliseconds, which may not be ideal for our scale.

Would be great to hear from the community if there are any alternatives to AES SIV which are faster (and deterministic), or if there are any other alternatives to AES encryption.

Here is my encryption method. I am using the Cryptodome.Cipher AES

    def encrypt(key: bytes, text: str) -> str:
        nonce = None
        cipher = None
        encoded_text = text.encode('utf-8')

        nonce = get_random_bytes(AES.block_size)
        cipher = AES.new(key, AES.MODE_SIV, nonce=nonce)
        cipher_text, tag = cipher.encrypt_and_digest(encoded_text)

        ret_cipher_text = nonce + cipher_text
        ret_cipher_text = tag + ret_cipher_text
        return b64encode(ret_cipher_text).decode()

The performance of this encryption ~2 millisecond for a small text.

  • $\begingroup$ 2 msec to AES encrypt a moderately small text? Surely, the CPUs on macbooks aren't that bad - I suspect you're using a bad (low performance) AES encryptor... $\endgroup$
    – poncho
    Jun 18 at 17:19
  • $\begingroup$ The phrase "deterministic" appears three times here. What is your specific concern? $\endgroup$
    – Paul Uszak
    Jun 18 at 17:25
  • $\begingroup$ @poncho - Yes, small text. I am using Cryptodome.Cipher AES. Are there faster alternatives? Possibly miscreant? $\endgroup$
    – Mandar
    Jun 18 at 17:34
  • $\begingroup$ @PaulUszak - I want to know if AES SIV encryption performance is indeed sub-optimal, and if there are any alternative ways to mask data such that the same plaintext gets masked to the exact same cipher-text and can be converted back to the plaintext. $\endgroup$
    – Mandar
    Jun 18 at 17:47
  • $\begingroup$ "or if there are any other alternatives to AES encryption." : You find find that (other than tls) there are no alternative encryptions from a marketing perspective. AES or bust. $\endgroup$
    – Paul Uszak
    Jun 18 at 18:00

The issue here is most likely Python and not AES.

With hardware support (AES-NI) AES can generally be computed at less than ~1 CPU cycle / byte if you have sufficiently many independent AES tasks (e.g. with counter mode) and at about 2-5 CPU cycles / byte if not (e.g. with CBC mode).

AES-SIV now effectively works by chaining a CBC-like operation with its output being used as the initial counter to counter-mode. Thus, the expected performance is about 3-6 cycles per byte for optimized implementations.

Two millisconds on a 1GHz CPU (yours probably clocks higher) are about 2 million CPU cycles. Even assuming a quite bad implementation on an older CPU with AES-NI a "bad" implementation will hit around 20-30 cycles / byte, but nowhere close to 2 million.

So the issue is most likely related to the interpreter having to do way more than with a compiled / optimized interpretation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.