As part of an experiment, I was given a .enc RSA (encrypted) file that was most likely generated using openssl. I had the public key, I factored it, obtained the private exponent, and decrypted the .enc file in chunks of 64 bit. What I got as a result, was nonsense hex. I was supposed to get a password or secret key.

What I have so far:

 Factors: 3917781347 x 4215069449 
 Private exponent: 9440767265896423601
 Modulus: 16513720463601767803

I have used xxd to extract the hex data from the ciphertext file: xxd -p > enc.hex

This created a hex dump of the file called 'enc.hex'. The contents of this file begin like this:


My Python code:

d = 9440767265896423601
N = 16513720463601767803

file = open('enc.hex', 'r')
c = file.read()
n = 16
c = c.replace('\n', '').replace('\r', '')
m = [hex(pow(int(c[i:i+n], 16), d, N)).rstrip("L") for i in range(0, len(c), n)]

f = open('decrypted.hex', 'w')

Where, 'c' is the ciphertext, 'd' is the private exponent, 'N' is the modulus and 'm' is supposed to contain the plaintext hex bytes.

Output of my python script:


When I convert this to ASCII, I get nonsense:


Also, '2F 2F' is not a Magic Number and does not indicate a file type. As I said, I was supposed to get a password one way or another after decryption.

I have spent weeks over this experiment, and I'm on the verge of crying now if I don't decrypt this and get the password to the next level.

Could someone please quickly verify what I've done is right, and they are getting the same hex sequence after decryption? Here are the files in a tarball:



  • .enc encrypted file
  • .uyu (was a hint that .enc is encrypted in RSA 64 bit chunks; you can ignore it)
  • .pem (a public key from where I extracted the modulus and public exponent)


I understand what I need to do now, and I know what to expect. However, my Python code seems to be messing up the 64 bit block decryption somewhere near the end as I'm not getting the two '==' signs (3d 3d) at the end after decryption. This is the trail (end) hexdump of my decrypted file:

   002ACD50  33 30 34 31  34 31 34 31   34 31 34 31  30 78 34 31   3041414141410x41
   002ACD60  34 31 34 31  34 31 34 31   34 31 34 31  34 31 30 78   414141414141410x
   002ACD70  34 31 34 31  34 31 34 31   34 31 34 31  34 31 34 31   4141414141414141
   002ACD80  30 78 35 61  35 31 33 64   33 64 34 31  34 31 34 31   0x5a513d3d414141
   002ACD90  34 31 30 78  30                                       410x0

The file unexpectedly ended with one byte (30). Could someone point out what is wrong with my python decryption code above?

  • $\begingroup$ Did you try to bsse64 decode this to binary? $\endgroup$ – Henno Brandsma Apr 6 '15 at 13:00
  • $\begingroup$ The = signs at the end are not necessary so try it anyway $\endgroup$ – Henno Brandsma Apr 6 '15 at 13:06
  • $\begingroup$ Write to file using > outputfile then do hd outputfile to see hex dump $\endgroup$ – Henno Brandsma Apr 6 '15 at 13:25
  • 1
    $\begingroup$ You are doing it right but perhaps there's a bug or a minor problem along the way. Note that your ciphertext is around 1.2 Mb long, so your plaintext (after RSA decryption) should be of the same size. Once you'll get this, decode the result with base 64 (you may need to pay attention to the end of the file, depending on how picky your base64-decoding tool is) and you'll end up with a playable MP3 file indeed. I am deliberately not posting source code as to not spoil your fun :) $\endgroup$ – Andrey Apr 6 '15 at 15:12
  • 1
    $\begingroup$ Please don't destroy your posts. $\endgroup$ – mikeazo Apr 7 '15 at 17:09

I did the procedure myself, and I can confirm that base64 decoding will do the trick; the full decrypted file will have == at the end somewhere. After decoding, the "file" command will reveal the plain text type. The '.uyu' file is quite easy to "decrypt" as well, I noted.

  • $\begingroup$ Remove the final 0x0. And also the final 4 41's (A's) as well, which are probbaly padding to fill out a block. The == (3d3d) are before that and are the end of the proper base64 $\endgroup$ – Henno Brandsma Apr 6 '15 at 18:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.