Cryptography Stack Exchange is a question and answer site for software developers, mathematicians and others interested in cryptography. It only takes a minute to sign up.
Sign up to join this community
Alice has a long message to send. She is using mono alphabetic substitution cipher. She thinks that if she compresses the message it may protect the text from single letter frequency attack by Eve. Does the compression help? Should she compress the message before the encryption or after the encryption? Defend.
Will compression help defeat single letter frequency attack against a mono alphabetic substitution cipher?
What user466720 said about ciphertexts being uncompressable is applicable to modern ciphers such as AES. It does not apply to monoalphabetic substitution ciphers, since such a cipher does not change the entropy of the data.
Per fgrieu's comment, you should assume that the attacker knows whichever compression algorithm is used.
Compression, unlike the monoalphabetic substitution cipher, does change the entropy of the data. When a monoalphabetic substitution cipher is applied to the compressed data, the encrypted data will therefore have the same entropy as the compressed data. As such, an attacker will not be able to conduct a frequency analysis on this data. That is because they do not know what the entropy is (which is a requirement of performing frequency analysis).
If you were to apply the cipher, then compressed the encrypted data, the attacker could decompress the encrypted data, and then perform frequency analysis on the decompressed data, assuming they can figure out the compression algorithm used. This is a fair assumption to make, since no aspect of the compression algorithm is secret and their specifications are freely available to all people.
The procedure for doing this is:
plain text $\rightarrow$ compression $\rightarrow$ encryption $\rightarrow$ cipher text
Since the cipher text is not compressible, we do compression before encryption. The secret key of a mono-alphabetic substitution cipher is not effected by this compression. So compression does not avoid single-letter frequency analysis, and mono-alphabetic ciphers can be easily broken by frequency analysis.
