Given the following conditions:

  • The encryption program, which uses some algorithm resembling AES-128 but with unknown modifications to real AES-128.
  • It uses a fixed key and IV.
  • It is written in Rust WITH debugging symbols but no source code is provided.
  • One is able to provide an input and get the encrypted output back.

Would it be possible to achieve the following?:

  • Decrypt let's say 48 bytes of ciphertext by just using a known 4 character crib?
  • Extract the key and IV from the program using gdb or similar. If so - how?
  • If the above is possible you can just decrypt any encrypted text, is that correct?

If the above is not possible:

  • Is it possible to reverse engineer the encryption given the conditions above? If yes, how?
  • $\begingroup$ @mentallurg it's a security question, but I'm thinking of cross-posting.. $\endgroup$ Aug 8, 2023 at 19:09
  • $\begingroup$ In a way it's also a programming question, because of the mentioned Rust part... $\endgroup$ Aug 8, 2023 at 19:53
  • 1
    $\begingroup$ @SirMuffington: I think there are two parts here: a) reverse engineering the algorithms with access to the compiled code, including extracting key and IV from code - that's basically decompiling and understanding what it does. b) treating the program as a black box with limited knowledge ("kind of aes-128, but unknown modifications, unknown key and IV") and probing it to find out the unknown parts - this is independent from the actual rust code or debug information -> better suited for Cryptography $\endgroup$ Aug 8, 2023 at 19:59

1 Answer 1


Decrypt let's say 48 bytes of ciphertext by just using a known 4 character crib?

I'm reading this as trying to use the encryption executable as a black box without reverse-engineering it, and iteratively modifying the plaintext so that it matches more and more of the given 48 bytes of ciphertext.

It's unclear why "4 character" would matter. AES encrypts 16 bytes at a time, and that's too wide for enumeration, unless there's a lot of redundancy in the plaintext (like, it's ASCII that was reencoded per UTF-32).

We can't tell if this would work, this depends on the mode of operation. It would not with CBC. It would with CTR or CFB, including with whatever variation of AES itself still making it a block cipher with 16-byte block size: for these modes, we can find the plaintext in 1 or 3 runs, respectively (and get a confirmation with an extra run).

But if we can run the executable, we can (and should) learn something on the mode used by exploring

  • The relation between plaintext size and ciphertext size.
  • What changing bit $j$ of the plaintext changes to the ciphertext.

Extract the key and IV from the program using GDB or similar. If so - how?

That's possible at least in principle, but can be of immensely varying difficulty depending on the effort that was made to make it hard.

At it's simplest, it's used unmodified AES-128 in a small variation of a standard mode (e.g. CBC with some custom padding that matters in the end only) and the key is a 16-byte segment of the executable. That makes it possible to mechanically test all such segments to find the key, and then finding the IV is easy. Perhaps it's necessary to change endianness, or/and perhaps the key is hex or Base64-encoded (which makes it easier to locate).

But the key could be arbitrarily encoded, or the algorithm could actually be a variation of AES (e.g. with a different choice of an arbitrary constant). Then it becomes necessary to reverse-engineer the code. That's off-topic, and requires experience. I'd try Ghidra in a VM. This would be on-topic on reverseengineering-SE.

Next difficulty levels would use encrypted code, interpretation, other obfuscation techniques, multiple layers of these. It's a game of hide and seek, in which the seeker is at a disadvantage that it must balance with good tools, experience, and effort.

If the above is possible you can just decrypt any encrypted text, is that correct?

Yes, for something AES-based, and more generally symmetric cryptography. That would not be the case for asymmetric cryptography, which makes it possible that even with full knowledge of the method and encryption key, it's impossible to decipher.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.