128 reputation
3
bio website
location
age 21
visits member for 2 years, 1 month
seen Oct 2 '13 at 2:21

Apr
10
awarded  Popular Question
Mar
21
comment 128 bit hash with least chance of collision
Hey Grant, thanks for your response. Is that negligible? I think it is. The probabilities for an accidental collision are still so low even with huge numbers of documents - as I said above, to the order of $10^{-15}$ with trillions of documents. Which, according to Wikipedia, is the unavoidable error rate in writing data to hard drives, and, more to the point, is unlikely to ever be noticed in my application. And the data is immutable, which solves the other problem.
Mar
19
comment 128 bit hash with least chance of collision
That was because I thought the truncated SHA you were referencing was more complicated than it was. Now I've reread things I've reached the same conclusion but for different reasons. I can't use a custom IV, which exposes what seems to me a potentially greater weakness than those of the Xor SHA512 solution - people can use standard rainbow tables and lookups to attack my keys to find near-collisions. With the Xor SHA512 method, it requires extra computation and strikes me as infeasible to attack.
Mar
19
comment 128 bit hash with least chance of collision
Thanks very much for your responses Mikeazo. I'm implementing this in PHP, Erlang and possibly Perl as well so I'd like as simple an implementation as possible. Which rules out truncated SHA and leaves Xor'ing SHA512 and @fgrieu's suggestion of RIPEMD-128. Between the two, I feel sure the Xor'ing SHA512 method isn't going to be defeated, whereas a weakness could be discovered in the lesser used and trusted RIPEMD-128. So I'm gonna go for the former. Do you agree?
Mar
19
awarded  Scholar
Mar
19
accepted 128 bit hash with least chance of collision
Mar
19
awarded  Student
Mar
19
comment 128 bit hash with least chance of collision
@mikeazo Thanks for the link. I'm afraid a lot of it flew over my head, but the 'near-collisions' looked bad, so how about this: I get an SHA512 hash and XOR each 128 bit chunk?
Mar
19
comment 128 bit hash with least chance of collision
@mikeazo I'm using PostgreSQL's UUID type, which seems the best way to efficiently represent the key. I'm worried about maliciously created collisions because, correct me if I'm wrong, even with trillions of documents hashed, the probability of a collision is still something like $10^{-15}$.
Mar
19
asked 128 bit hash with least chance of collision