For ElGamal-based key encapsulation, is it necessary to hash before using as AES key?

Question

I'm working with SJCL, specifically using ElGamal to encrypt messages. Behind the scenes, this is doing something similar to what's described in this SO post (emphasis mine):

Regardless how big prime p is, you can always generate a random m number in the range of 1 to p-1 and use that to produce your asymmetric ciphertext. Afterwards, you can take the previously generated m, encode it into a byte array (use toString(16) to produce a Hex-encoded string and then simply parse it as Hex for the hashing) and hash it with a cryptographic hash function such as SHA-256 to get your AES key. Then you can use the AES key to encrypt the message with a symmetric scheme like AES-GCM. This is called key encapsulation.

Here is the actual function:

kem: function(paranoia) {
    var sec = sjcl.bn.random(this._curve.r, paranoia),
        tag = this._curve.G.mult(sec).toBits(),
        key = sjcl.hash.sha256.hash(this._point.mult(sec).toBits());
    return { key: key, tag: tag };
  }

My question is: regarding key = sjcl.hash.sha256.hash(this._point.mult(sec).toBits()), is it necessary to compute SHA256 hash and use that as AES secret? Or can we get by with something like this: key = this._point.mult(sec).toBits() (no hash)? Would this be less secure?

I ask as I am trying to take advantage of multiplicative homomorphism of ElGamal. So, multiply encrypted secret by a scalar, decrypt new ciphertext, and multiply decryption result by scalar inverse to get original secret. I am able to do this when I remove the SHA256 hashing code above, however with hashing code still enabled I am getting a not on the curve error when trying to multiply decryption result by scalar inverse (because we are using hash for key which I believe pushes it off the curve).

So, wondering if removing hash logic degrades security of the method / is not advised or if hashing is performed for different reasons (and wondering what those reasons are if so).

Answer 1

The issue here was that the hashing is used to derive an AES key for encryption from the value $m$ in the question. However, it is still the value of $m$ that needs to be encrypted using the public key and the homomorphic ElGamal algorithm, not $H(m)$. After homomorphic decryption the value $m'$ can be retrieved, and $H(m')$ will create the same AES key to be used to decrypt the message iff $m' = m$.

The idea of the hash is to compress the entropy in $m$ so that the (leftmost bytes) of the result can be used as a well distributed AES key.

Even more information and the road towards this answer can be found in the comments below the question and the chat.