I am creating an end-to-end encrypted group messenger, and am looking for a way for online members of the chat to a arrive at a shared secret that can be used as a cryptographic key. This will probably happen when someone joins or leaves the chat, and all members of that chat have the public keys (RSA-2048 bit) of all the other members, online or not. The relay should not know, be able to derive this shared key, or be able to manipulate the key exchange process in any way that would allow it to do either of the above. All communication must happen through the relay.
Currently, the relay selects one of the online members of the group (admin if possible) to generate the key, hash it, sign the hash, and encrypt the key with all the other members public keys, which the relay then passes on to them, and if they all accept the new key (e.g hash and signature is correct), the relay issues the key switch command, but this requires five stages, so I was wondering if there was a more efficient way, or if my current way has any security flaws.