You seem to (incorrectly) describe PKCS#7 compatible padding.
PKCS#5 (compatible) padding is the same as PKCS#7 (compatible) padding, but defined only for block ciphers of 8 bytes. The current PKCS#5 specification (which defines Password Based Encryption and defines a padding method because it wasn't standardized yet) is even referencing PKCS#7 for the padding when a 16 byte block cipher is being used.
PKCS#7 doesn't pad with characters such as 2
(byte value 0x32
in ASCII) but it pads with bytes with value 0x02
when two bytes are required.
When a program is decrypting a message encrypted by CBC mode, what does it find out, whether the last bytes of the last block is part of the padding or part of the plaintext?
First of all, the padding mode needs to be established beforehand. It is part of the scheme being used. To be very precise, it is really more of a configuration option for ECB / CBC as other modes don't require padding at all. Generally it is part of the protocol description, not a configurable parameter.
Secondly, PKCS#7 padding is always applied. There is always at least one padding byte added. Furthermore, it is only needed to pad up to the next block. So there are always 1 to N padding characters, with a byte value of 1 to N respectively. During unpadding first the last byte is taken, call this value P. If it is not within the value 1..N an error is generated. If the previous P - 1 bytes do not have the correct padding value of P then the same error is commonly generated.
This also means that if the plaintext is already a multiple of N bytes that a full block of padding must be generated, all with value N. For AES that means a full block with value 10101010101010101010101010101010
in hexadecimals.
So you find out by unpadding without looking at the code. You only need to inspect the decrypted block if you don't know the padding in advance.
Sometimes, to avoid padding oracle attacks, no error is generated. In that case the plaintext might itself contain a MAC. This is for instance the case for TLS connections prior to TLS 1.3 (where this dangerous construction was removed entirely).