I'm having a lot of trouble trying to understand how to implement AES-GCM.

My usecase: Person A sent me a file with a gauranteed unique name. I want to be assured it was from person A, and that no one saw the contents, and that it wasn't tampered with.

I figured that I would do the following:

  1. Person A would give me a public key offline to which only they have the private key
  2. Person A would encrypt the file with AES-GCM. They would then encrypt the AES-GCM key+nonce+aad with their private key. They would then send me two files: the cipher and the encrypted key+nonce+aad combo.
  3. I would decrypt the key+nonce+aad with their public key, then I would decrypt the AES-GCM cipher using those values.

I figured that I would use these values:

  • Key: random
  • Nonce: file name
  • AAD: file name

I know this is probably a terrible way to do things. It seems overly complicated to me, but I'm not sure how else to achieve my usecase. I have no idea if these are the right values to use for the encryption/decryption, and I have no idea what should be kept secret and what shouldn't be kept secret.

Any guidance would be greatly appreciated. If it helps, it looks like the GSM encryption/decryption will happen in Python.

  • $\begingroup$ Maybe it should be that the key is random, the nonce is the filename and the AAD is the key encrypted using the private key. The cipher, nonce and AAD are passed along to the recipient. The recipient decrypts AAD with their public key to get the cipher key and hence is able to decrypt the cipher. Is this more terrible or less terrible? $\endgroup$
    – Kias
    Oct 6, 2015 at 20:20
  • $\begingroup$ you don't need to PK-encrypt the AAD and the nonce... You can prepend the nonce to the cipher text (it's always part of the AAD) and you can basically add anything you know about the file and is preserved in transit to the AAD. Furthermore you don't need to explicitely transmit the AAD but may prefer leaving it implicit. $\endgroup$
    – SEJPM
    Oct 6, 2015 at 20:35
  • 3
    $\begingroup$ In fact, typically the point of the AAD is to be provided implicitly as context rather than being something entirely attacker-controlled. Also, the filename being used as the nonce is extremely bad — nonces must never repeat, and if you ever reuse a filename or send the same file with a few modifications, the entire security of the system collapses. $\endgroup$ Oct 6, 2015 at 21:25
  • $\begingroup$ Also, it should be said that if you're implementing this as a toy project for the sake of learning, great! If you intend to ever use this in a real-world system, just use GPG. $\endgroup$ Oct 6, 2015 at 21:26
  • 1
    $\begingroup$ It's intended as a real world system. And I mentioned the filenames were guaranteed unique. My use case is not what I stated.. it's actually much different, but I simplified my actual scenario for the question. $\endgroup$
    – Kias
    Oct 8, 2015 at 4:19

2 Answers 2


Typically things would go like this:

  1. A generates a random AES key and encrypts it with the public key.
  2. A encrypts the file contents using AES GCM and that key. Nonce can be random or even zero if the key is only used once. AAD can be empty unless something else needs to be authenticated with the file.
  3. A sends over the encrypted key, the nonce if any, and the ciphertext.

The file name only matters if you need to encrypt it too (simplest: encrypt a tar archive instead) or if you need the name authenticated (include it in the AAD).


What your are describing is the typical use for public key cryptography. It is usually refereed to as KEM/DEM or Hybrid Paradigm. KEM/DEM stays for Key Encapsulation Method, Data Encapsulation Method.

To encrypt a file one draw a random secret ket to be used with AES, he draws also a random IV and a nonce and potentially an AAD. He use this data to encrypt the file under the AES-GCM algorithm and then encrypt the secret key under your public key. He finally sends you the encryption of the secret key and the encrypted file.

You use your private key to decrypt the secret key used with AES-GCM and then decrypt the file.

In your question you mistaken the use of public and private key.

Also to be sure that the sender is Person A and not someone else he should sign the message, using his (public, private) key-pair. Other way there is no way to prove that the encrypted file was actually generated by someone in particular.

  • 1
    $\begingroup$ Doesn't KEM imply using the asymmetric primitive and deriving the key rather than encrypting a random key as done here? $\endgroup$
    – otus
    Oct 7, 2015 at 7:47
  • $\begingroup$ I never heard about this way to use the KEM/DEM paradigm. And I cannot easily see how you can derive a key using an asymmetric primitive: both parts need the same symmetric key to encrypt/decrypt the message. How can they derive the same key from an asymmetric primitive, given that one has only the public part? that would mean that an attacker could recover it too. $\endgroup$
    – ddddavidee
    Oct 7, 2015 at 7:57
  • $\begingroup$ Sorry, my comment was unclear. I meant that with KEM you do not encrypt the key, you derive the key from a random group element. $\endgroup$
    – otus
    Oct 7, 2015 at 7:59
  • $\begingroup$ Sorry, I misunderstood your comment. $\endgroup$
    – ddddavidee
    Oct 7, 2015 at 8:01
  • $\begingroup$ I checked the Canetti paper and he use the definition you gave, but I've to admit I heard KEM/DEM as a synonym for hybrid encryption, maybe it is an language abuse and I should correct the answer. $\endgroup$
    – ddddavidee
    Oct 7, 2015 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.