I have a chat server which uses public keys to identify their clients. But if the public key is known, then everyone can send a direct message (DM) to each other, which is annoying. The clients should choose who should DM them.
So I decided to get two hashes of the public key, one for user authentication and one for identifying the public key used to encrypt the direct messaging.
My question is: does having two hashes affect the probability of successfully guessing the original data?
IMO it depends on what you mean by the probability of successfully guessing the original data. Since hashing functions turn input of any size to a fixed-size output, there are (infinitely) many collisions. That may pose a problem with passwords, where any collision results in successful authentication. However, you only care about the correct preimage, as any other input, that produces the same output, (most likely) does not reveal Bob's valid public keys.
Suppose there is an attacker who knows your public keys format and is able to brute-force all possible keys. Then it is likely that more than one candidate will appear (depending on the length of your hash and public keys). The attacker then gets more information from the second hash as to distinguish which candidate is the correct one.
In that sense, you increase the probability of guessing the original data by providing a second hash.
But. I don't think that scenario is much to worry about - as you need the collisions of the first function to feed into the second function to gain advantage (or vice versa) and these are hard to get if the hashing function is well-chosen. Moreover, given that the number of such candidates might not be too high, the attacker might just as well try to DM using candidate keys rightaway.
If you worry about someone finding correct preimage of a given hash (or two), use properly-sized and random salt, different for each user. Ultimate booster: use different salts for each hash.
