Cryptography Stack Exchange is a question and answer site for software developers, mathematicians and others interested in cryptography. It only takes a minute to sign up.
Sign up to join this community
When a program is decrypting a message encrypted by CBC mode, what does it find out, whether the last bytes of the last block is part of the padding or part of the plaintext?
Example: Use CBC mode with 4 Byte blocksize and encrypt the following two messages:
m1 = ABCDEF
m2 = ABCDEF22
M1 gets padding and will be "ABCDEF22". How do I find out, whether the original plaintext was ABCDEF, or ABCDEF22?
Actually, that is depending on the padding applied. Maybe there is no padding at all. The implementation defines the padding applied. Check the code or documentation!
The last 22, at first, indicates that might be PKCS#5 or PKCS#7 since PKCS#7 most common with CBC implementations. While PKCS#5 is limited to 8 bytes like DES, PKCS#7 works from 1 to 256 bytes. PKCS#7 appends octets like
For such algorithms, the method shall be to pad the input at the trailing end with $k - (l \bmod k)$ octets all having value $k - (l \bmod k)$, where $l$ is the length of the input.
However, that is 0x02,0x02. The 2s in your last 22 is the character 2 that is in hex 0x32.
To see that has PKCS#7 padding check the last octet and maker sure that the value $k$ of the last octet repeat $k$-times. Example for a 16-byte block cipher where each position is a byte. 1 means 0x01, 2 means 0x02, etc.
xxxxxxxxxxxxxxx1
xxxxxxxxxxxxxx22
xxxxxxxxxxxxx333
...
xFFFFFFFFFFFFFFF
xxxxxxxxxxxxxxxx full block append a new block that contains 16 times 0x10
To see other padding candidates see Padding in Wikipedia
You seem to (incorrectly) describe PKCS#7 compatible padding.
PKCS#5 (compatible) padding is the same as PKCS#7 (compatible) padding, but defined only for block ciphers of 8 bytes. The current PKCS#5 specification (which defines Password Based Encryption and defines a padding method because it wasn't standardized yet) is even referencing PKCS#7 for the padding when a 16 byte block cipher is being used.
PKCS#7 doesn't pad with characters such as 2 (byte value 0x32 in ASCII) but it pads with bytes with value 0x02 when two bytes are required.
When a program is decrypting a message encrypted by CBC mode, what does it find out, whether the last bytes of the last block is part of the padding or part of the plaintext?
First of all, the padding mode needs to be established beforehand. It is part of the scheme being used. To be very precise, it is really more of a configuration option for ECB / CBC as other modes don't require padding at all. Generally it is part of the protocol description, not a configurable parameter.
Secondly, PKCS#7 padding is always applied. There is always at least one padding byte added. Furthermore, it is only needed to pad up to the next block. So there are always 1 to N padding characters, with a byte value of 1 to N respectively. During unpadding first the last byte is taken, call this value P. If it is not within the value 1..N an error is generated. If the previous P - 1 bytes do not have the correct padding value of P then the same error is commonly generated.
This also means that if the plaintext is already a multiple of N bytes that a full block of padding must be generated, all with value N. For AES that means a full block with value 10101010101010101010101010101010 in hexadecimals.
So you find out by unpadding without looking at the code. You only need to inspect the decrypted block if you don't know the padding in advance.
Sometimes, to avoid padding oracle attacks, no error is generated. In that case the plaintext might itself contain a MAC. This is for instance the case for TLS connections prior to TLS 1.3 (where this dangerous construction was removed entirely).
