Encrypting many small messages (log-lines)

Question

What would be the best way of encrypting small mesasages to be stored in encrypted form? (This sounds like somthing a textbook would cover, but I haven't found any references).

Let's say I have many systems producing messages (log-lines) which I would like to transfer and store centrally in a secure manner. Using off-the-shelf products I could secure the transfer and storing of these messages independently, but I would like to investigate another approach: encrypting each message at the sendig system with the public key of the reciever, so that the messages can be stored as-is, and thus provide end-to-end encryption from the sender down to the storage medium.

First some definitions:

• RSA(PublicKey, Message) means: RSA-encrypt Message with the given Key
• AES(Key, IV, Message) means: AES-encrypt Message in CBC-mode with the given Key and IV
• Message = gzip(plaintext)

I have come up with the following options:

I can encrypt each message with RSA:

• Ciphertext = RSA(PublicKey, Message)

However, this would fail of the message is too long.

Or I could create a random AES-key for each message (and also use this as IV):

• Key = Random()
• EncryptedSessionKey = RSA(PublicKey, Key)
• Ciphertext =EncryptedSessionKey, AES(Key, EncryptedSessionKey, Message)

However, RSA-computation is heavy and I probably cannot afford to do this for every message. Thus, I can re-use the key for, say, 100 messages and do:

• Key = Random() (re-key every 100 messages)
• EncryptedSessionKey = RSA(PublicKey, Key)
• IV = Random() (new IV for each message)
• Ciphertext = EncryptedSessionKey, AES(Key, IV, Message)

The last one seems to be both secure and efficient. However, I really don't want to invent a system like this myself. Is there any best practices, research or textbook-examples I should look into?

Answer 1

Ok, answering myself then, trying to summarize all the good input from above.

First of all, this design will provide confidenciality to each log message, but it will not:

• ensure the integrity of the log as a whole, it will be possible to insert or remove log-entries unnoticed
• authenticate the sender of log-messages, a rouge client or a man-in-the-middle could insert fake log-messages

However, if this is OK (or you have other ways of solving these issues), this scenario is probably secure.

As for performance, on my machine I get the following numbers:

• Just RSA: just below 4000 messages pr second
• RSA and AES with a new key each time: just above 4K msg/sec
• RSA and AES with a new key each 10 messages: 17 K msg/sec
• RSA and AES with a new key each 100 messages: 25K msg/sec

Just RSA seems to have no advantage over RSA+AES, and there is an upper limit to message size.

Now, this just encrypts the same string over and over. For a more real-life test I will make new strings (serialized xml) and write these to disk, to se if the difference is still significant.

As for the use of gzip, I have tried a 218-byte xml-message which gzipped into 164 bytes (25% reduction in size). This is a relevant testcase in my scenario, but of course your milage may vary.

Edit: I have re-tested with different values for key-size and exponent. All of these are using RSA to encrypt an 128-bit AES session key, with a new key for each message (I guess in effect I am measuring the throughput of RSA-encryption of 128-bit random numbers).

• keysize=1024 bits, $e = 2^1+1$: 7 tx/millisecond
• keysize=1024 bits, $e = 2^{16}+1$: 10 tx/millisecond
• keysize=2048 bits, $e = 2^1+1$: 11 tx/millisecond
• keysize=2048 bits, $e = 2^{16}+1$: 4 tx/millisecond

This is using the default crypto-provider in java 7. I let each configuration run 20 iterations before starting the measurement to "warm up" the jit-compiler.

My environment:

• java.vm.version: 23.7-b01
• java.vm.vendor: Oracle Corporation
• java.vm.name: Java HotSpot(TM) 64-Bit Server VM
• file.encoding.pkg: sun.io
• java.runtime.version: 1.7.0_15-b03
• os.arch: amd64
• os.name: Windows 7
• sun.management.compiler: HotSpot 64-Bit Tiered Compilers
• os.version: 6.1
• sun.arch.data.model: 64