Data I/O operations for encrypted files

Question

I need to implement a simple approach on a Linux system to encrypt/decrypt data and I would appreciate any feedback from you. Basically, I need to change a bit the behavior of the functions read/write.

I've checked some options such as EncFS, but they only perform encryption/decryption at file level (decrypt/encrypt the entire file at once). However, I need some solution at a fine-grained level, such as at block level. To make things easier, we only append data at the end of the file and we don't change existing data of the file.

Consider the following use case, as an example:

• File foo.txt is empty
• Write 10 bytes to file foo.txt
  • Encrypt the input (10 bytes) and write it to the file foo.txt. Depending on the encryption algorithm, the plaintext and ciphertext sizes might differ.
• Read 6 bytes from file foo.txt
  • How should you handle this situation? Should you decrypt everything you have and extract only 6 bytes?

What I have seen is that you most likely have to keep also metadata about the encrypted files as well (number of blocks, etc.). Thanks in advance!

Answer 1

Ad hoc…

What you’re describing is not much different from the encryption in storage systems… only that your “storage system” is a file (foo.txt) instead of a hard drive. Now, in a perfect world you would encrypt every block of data (that you store in foo.txt) with its own key, but that would be pretty impractical in your situation. From my point of view, the most obvious solution would be to use a “tweakable cipher” or a “tweakable narrow-block encryption mode”. They were designed to avoid (read: work around the problem) the “need separate key per stored data block”.

Therefore, and relying on your description, I’ld most probably opt-in for a “tweakable narrow-block encryption mode” like LRW (paper), XEX (paper), or XTS. An example representing one of the more frequently used implementations would be AES-XTS. Depending on your coding skills, it shouldn’t be all too hard to incorporate such a solution into the CRUD functionalities of your application.

But, it depends on the details…

For the sake of completeness, let me mention that there are other options out there, which might also lead to a valuable solution. I merely mentioned what I personally would tend to use, based on the information you provided. But choosing a fitting solution will strongly depend on things like the kind of data you want to store, how it’s formatted, what security level you expect, what specific scenario is to be assumed, etc. (Always check for potential attack vectors!) You will have to assess and verify all the little details to be able to choose a most optimal solution that perfectly fits your individual situation and needs.

One of many examples: If the filesize and available hardware capabilities permit it, you might be better off handling encryption/decryption on the whole file (using AES-CBC or something like that) instead of crypting the individual data-blocks you plan to create, read, update and delete. On the other hand, if we’re talking about a file that is bound to hold hundreds of thousands of individual data blocks/entries, and which therefore is bound to grow up to and beyond several gigabytes in filesize, loading that beast into system memory to wrangle with contained data will tend to be a bit… (let’s just call it) “non-optimal”.