Implementing PKCS#7 Padding on a Stream of Unknown Length?

Question

I have a fairly simple Python program using PyCrypto to use AES+CBC to encrypt a stream of input. In order to adhere to the 16-byte input size multiple, I've implemented PKCS#7 by hand. (While I know implementing cryptographic methods by hand is generally a bad idea, padding 1-16 bytes of data isn't exactly the hardest thing in the world. Here is essentially what I'm doing:

while True:
    input_chunk = infile.read(1024)

    if len(input_chunk) == 0:
        # if we've reached the end of the file and it _is_ a 
        # multiple of 16 in length, pad 16 bytes with the value '16'
        end_of_line = True
        input_chunk += struct.pack('{}B'.format(16), *(16 * [16]))
    elif len(input_chunk) % 16 > 0:
        # if we don't have an input_chunk which is divisible by 16,
        # pad it by the remainder with bytes with the value of the
        # remainder
        end_of_line = True
        input_chunk_remainder =  16 - (len(input_chunk) % 16)
        input_chunk += struct.pack('{}B'.format(input_chunk_remainder),
                *(input_chunk_remainder * [input_chunk_remainder]))

    outfile.write(encryption_cipher.encrypt(input_chunk))

    if end_of_line:
        break

The only problem I'm now having is trying to understand how I'll decrypt an input stream if I don't know its length in advance. The encryption I'll be implementing will actually be used to read a stream of bytes from a socket, decrypting each 'chunk' as it comes across.

How will I know when to trim the PKCS#7 padding? Can I assume that if the last n bytes in the stream are all the same value from 1-16 that I can trim the input after decrypting it? I would assume that this is dangerous and may trim data unnecessarily.

Is there a way to solve this problem?

Answer 1

Bit padding, PKCS#5 padding and PKCS#7 padding are always applied, and they are always applied to the end of the plaintext. The last block of ciphertext will always contain the padding. The padding is always 1 to [blocksize] bytes, which in case of AES is 1 to 16 bytes.

For PKCS#5 padding and PKCS#7 padding - which are basically identical - the value of the padding bytes is equal to the length of the padding. So you would get values 10, 0F, 0E to 01 (in hexadecimals) for ciphertext lengths Z*16 + 0, 1, 2 to 15 respectively.

To unpad quickly simply read the last byte of the decrypted ciphertext, and remove that number of bytes (not characters) from the end of the plain text.

Notes:

• It is possible to verify the other padding bytes, as long as the padding bytes themselves are not used to verify the correctness of the data;
• It is only possible to unpad at the end of the ciphertext but it is not possible to use the padding bytes to detect the end of the ciphertext (unless the padding is clearly distinct from the plain text);
• Please know of and beware of padding oracle attacks.

Answer 2

Have you considered Stream Ciphers? They seem better suited for your problem.