So I am looking for an explanation of an experiment. In this experiment, I took a set of k hash functions. Say the total number of data points I am working on is d. Call an algorithm A which used that set of hash functions to do some operations on d data points and have modified them to a matrix B.It is mentionable that algorithm A introduces a bunch of randomization. So applying algorithm A different times with same set of hash functions and same set of data points will produce different results. Now we apply algorithm C on that matrix B and get a result, call it D. So this is the summary of what I am trying to do in short. Now I can do this multiple times in two ways-
- I take a set of k hash functions in the beginning. Now I run algorithm A with the same set of hash functions and on the same set of data points n times and produced n different matrixes and averaged the respective elements of those matrixes to produce a final matrix, call B'. So in short, the (i,j)th entry of the matrix B' is the average of (i,j)th entry of n matrixes we got before. Now , we run algorithm C on matrix B' and get a result D
- I take a set of k hash functions, run algorithm A, generate matrix B and then run algorithm C on matrix B to produce result D. Now, I run this process multiple times, every time with a different set of hash functions and then average all such Ds and produce D'
Interestingly, process 2 gives just radically more accurate results and I don't know how to explain this behavior. One thing to mention, I was using sha256 in this experiment.
I hope to get help from kind people out here in order to solve this.
Thanks in advance!
Edit: This has been used for some differential privacy operations and the results accuracy is being measured by comparing with the ground truth
Edit2: By indicating that I used k different sha256 hash functions, I meant that I just generated k seeds and appended them before the string I am tryna hash, for example,
hex_num = hashlib.sha256((str(seed[i])+str(d)).encode('utf-8')).hexdigest()
Hope it is clear now