# For ElGamal-based key encapsulation, is it necessary to hash before using as AES key?

I'm working with SJCL, specifically using ElGamal to encrypt messages. Behind the scenes, this is doing something similar to what's described in this SO post (emphasis mine):

Regardless how big prime p is, you can always generate a random m number in the range of 1 to p-1 and use that to produce your asymmetric ciphertext. Afterwards, you can take the previously generated m, encode it into a byte array (use toString(16) to produce a Hex-encoded string and then simply parse it as Hex for the hashing) and hash it with a cryptographic hash function such as SHA-256 to get your AES key. Then you can use the AES key to encrypt the message with a symmetric scheme like AES-GCM. This is called key encapsulation.

Here is the actual function:

kem: function(paranoia) {
var sec = sjcl.bn.random(this._curve.r, paranoia),
tag = this._curve.G.mult(sec).toBits(),
key = sjcl.hash.sha256.hash(this._point.mult(sec).toBits());
return { key: key, tag: tag };
}


My question is: regarding key = sjcl.hash.sha256.hash(this._point.mult(sec).toBits()), is it necessary to compute SHA256 hash and use that as AES secret? Or can we get by with something like this: key = this._point.mult(sec).toBits() (no hash)? Would this be less secure?

I ask as I am trying to take advantage of multiplicative homomorphism of ElGamal. So, multiply encrypted secret by a scalar, decrypt new ciphertext, and multiply decryption result by scalar inverse to get original secret. I am able to do this when I remove the SHA256 hashing code above, however with hashing code still enabled I am getting a not on the curve error when trying to multiply decryption result by scalar inverse (because we are using hash for key which I believe pushes it off the curve).

So, wondering if removing hash logic degrades security of the method / is not advised or if hashing is performed for different reasons (and wondering what those reasons are if so).

• The hashing was used to create an AES key. You seem to be using differently. Maybe you can explain what you are trying to achieve? Currently you describe what you are doing, but that's not all that clear (to me). The part about hexadecimal encoding seems completely off topic by the way. – Maarten Bodewes Aug 27 '18 at 22:51
• Sorry, I included that just to show another approach that also uses hashed value as AES key as opposed to raw. Why do we need to hash to create AES key? Bear in mind my knowledge is limited, but I'm trying to be able to manipulate the keys generated by this function. When I hash them it seems they "fall off" the secp256k1 curve and I can no longer multiply them properly. – pgorsira Aug 27 '18 at 22:59
• I want a user to be able to take a key that's been encrypted by an ElGamal public key (tag in code above), multiply by scalar val, and have another party decrypt it (without knowing what they're decrypting as it doesn't look like the original ciphertext), and then be able to convert this decrypted text to original plaintext (by dividing by scalar). Does that make sense? – pgorsira Aug 27 '18 at 22:59
• Kind of. But note that ElGamal may be homomorphic. Unfortunately, the symmetric encryption in a hybrid cryptosystem isn't, and you cannot magically make this so. If there is such a scheme then it is certainly beyond my knowledge. – Maarten Bodewes Aug 27 '18 at 23:11
• Right, so when I say "ciphertext" I'm talking about the encrypted AES key (it's encrypted with ElGamal public key), not the actual message encrypted by the AES key. I don't require any kind of homomorphism with respect to the symmetric encryption. The example above is purely dealing with AES keys themselves, not the data encrypted with those keys. – pgorsira Aug 27 '18 at 23:14

## 1 Answer

The issue here was that the hashing is used to derive an AES key for encryption from the value $m$ in the question. However, it is still the value of $m$ that needs to be encrypted using the public key and the homomorphic ElGamal algorithm, not $H(m)$. After homomorphic decryption the value $m'$ can be retrieved, and $H(m')$ will create the same AES key to be used to decrypt the message iff $m' = m$.

The idea of the hash is to compress the entropy in $m$ so that the (leftmost bytes) of the result can be used as a well distributed AES key.

Even more information and the road towards this answer can be found in the comments below the question and the chat.