Is storing original file size in an encrypted file header a bad idea?

Question

I'm encrypting a file using AES-256 in CBC mode. I'm padding to the 16-byte input multiple by using the PKCS#7 limit. The problem I'm currently having is that if I'm transferring or reading the encrypted file over a socket, I won't know when the encrypted file has been finished, as I'm reading it in chunks. Because of this, I won't know when to trim the PKCS#7 padding from the decrypted file chunks. (I won't know the size of the decrypted data in advance, so I won't know where to look for padded bytes and to trim them.)

One solution I've thought of would be to include the unencrypted file size in the header of the encrypted file, but I'm not sure if this is a security threat or not. Is this a bad idea?

Edit

I have kind of a strange setup. Encrypted data is stored like so:

$$s \longrightarrow e_{m} \longrightarrow d$$

The source data $s$ is passed to $e_{m}$, the encryption method, over a socket. As $e$ encrypts the data, it simultaneously writes the data over another socket to the destination $d$.

Retrieval works in the inverse:

$$d \longleftarrow d_{m} \longleftarrow s$$

The encrypted source $s$ is passed through the decryption method $d_{m}$, which decrypts it and passes the decrypted data to destination $d$.

With this in mind, how can I maintain padding with AES? The contents being transferred are files, so they are inherently and usually larger than 50 bytes of data. The problem seems to be $e_{m}$ and $d_{m}$ not knowing the size of the input data and simply encrypting block-by-block.

How can I create a decryption method which, not knowing the size of $s$, will still be able to decrypt the data and send it to $d$?

Answer 1

The 16-byte IV and ciphertext (which together are part of the output of $e_m$) are assumed to be intercepted by an adversary. That reveals the number $b$ of 16-byte blocks in the ciphertext. With CBC and PKCS#7 padding, $b=\big\lceil{{n+1}\over16}\big\rceil$ where $n$ is the byte size of the plaintext (the file size). Putting $n$ itself in a header thus reveals a little extra information: $n\bmod16$, which is worth 4 bits. In some cases, e.g. if what's encrypted is yes ($n=3$) or no ($n=2$), this is enough to ruin security. Thus no, it is not always safe to include the original file size in clear. But in practice, it is often assumed that the length of the data can be public; then it can be sent in clear.

There's a fix to solve the networking problem as originally exposed while revealing no more information about the length of the plaintext than implied by the use of CBC and PKCS#7 padding: in the header, put the number of blocks $b$ in the ciphertext (not counting IV), or equivalently $\big\lceil{{n+1}\over16}\big\rceil$, where $n$ is the size in bytes of the file to encipher.

But caveat emptor: the above fix is mostly of theoretical interest. While it maintains security in the yes or no case, it does not work for my_answer_is_yes ($n=16$) versus my_answer_is_no ($n=15$); or if the plaintext is a JPEG image known to be either out of focus (thus quite compressible, thus small), or well-focused (thus less compressible, thus bigger). Most encryption modes leak a lot information about the length of the plaintext, and this can be exploitable. This effect can be the basis of attacks on VOIP encryption. In most cases where the size of the data reveals information to an adversary, standard operating modes such as CBC are just inadequate.

Update following edit of the question: the networking problem as now explained could be solved by having $b$ computed from $n$ and sent in clear at the beginning of each of the 3 streams involved; the PKCS#7 padding performed by the process reading the file and sending it to $e_m$ (not by $e_m$); and the PKCS#7 un-padding performed by the process retrieving the output of $d_m$ (not by $d_m$). But then the block size (16 bytes) needs to be known by all 4 processes involved, when we would want to isolate that detail, and padding, to the two processes $e_m$ and $d_m$.

It is also possible to have the file size as the first block of the plaintext, CBC-encrypted as the rest. This allows to have everything regarding encryption, its block size, etc.. isolated to $e_m$ and $d_m$. If the length is encoded in a single dedicated block (e.g. the first of the plaintext, thus the one XOR-ed with the random IV), it does not leak more information about the size of the plaintext than implied by CBC-encryption with PKCS#7 padding, just as in the original fix.

Note: I now realize the title mentions an encrypted header, and the question mentions an unencrypted file size in the header. I've been considering the later in all except the previous paragraph.

Note: There is nothing to protect integrity of the data; that is often a very real problem.

Answer 2

Generally sockets (or at least, sockets that use some kind of transport layer like TCP) get closed. So you are at the end when the stream of data is at its end. So you do the unpadding then. Except that without MAC or authenticated mode of encryption you are extremely vulnerable to e.g. padding oracle attacks.

If you want to send the size of the data in the clear, you could use an authenticated mode of encryption (such as GCM or AEX as indicated in other questions) or a MAC and include it in the calculations. If an attacker changes the size your MAC verification will fail, and you can reject the ciphertext.