1
$\begingroup$

I'm looking for a way of one way hash data for multiple records. We have a privacy requirement to anonymise information "after it's used for its intended purpose" which is a little wooly at best.

I'd therefore like to delete everything once a project is finished, but this has been met with pushback internally "in case clients as follow up questions", so I figure the next best way is to hash the data with a one way system, so, it's effectively anonymized, not even we'd know what it was, but, if supplied with an email address (for instance) to lookup, we could hash it and search for a match in our database.

I'd need something that non-technical users could use and wouldn't be overly onerous in time, as it's an uphill battle to get people compliant already.

Hoping for an excel solution, or something that could load an excel/csv in and spit out the hashed version, however everything I look at for PBKDF2 is for implementing in websites or similar, singular password use.

I managed to get SHA256 working in excel, but as there's no salt they can be reversed with some work, meaning our effort to make the data anonymised is disingenuous, which is where i came across PBKDF2.

As you can tell be reading this far, I'm a bit out of my depth and would like to appeal to those who know more than I for any help!

Improve this question
$\endgroup$
8
  • $\begingroup$ "I managed to get SHA256 working in excel, but as there's no salt they can be reversed with some work" ? Isn't that quite tricky? How long is each record? $\endgroup$
    – Paul Uszak
    Commented May 2, 2019 at 14:00
  • $\begingroup$ We'll be hashing personal data, so names, email addresses etc.. so fields could be four to over forty characters. It's less of a practical thing though and more of the theoretical, so we can check a box to say we've done this to the best possible way, if you get me? $\endgroup$
    – Sam
    Commented May 5, 2019 at 23:30
  • $\begingroup$ I do and don't :-( but happy to help :-) PBKDF2 is used for reversible encryption, but that's neither here nor there. My problem is your 1st & 2nd paragraph. You seem to require quasi anonymous data which seems less than anonymous. But anonymity is a binary condition. And reversing SHA256(favourite colour|name|email) is currently impossible. I think that you need to flesh out paras.1 & 2 as cryptography is not anonymity. Asking the right question is 50% of the answer... $\endgroup$
    – Paul Uszak
    Commented May 6, 2019 at 0:52
  • $\begingroup$ Sorry Paul. So I want anonymity in the regards that if you don't know the answer you can work it out (so, say rainbow tables could give you the result of a basic md5 hash pretty easily). But, if we have an email, say [email protected] and a salt of [email protected] (unique salt) for 1k iterations & key length 32 you will get the hash: E4AE2A011E2B6454B384B33EB55E8BFAAE057D7C60C872C5F2F2E25BCC90C76A Now, it'd be pretty difficult to reverse that, but, in the future if someone goes, "did you contact [email protected]" I can re-apply the above process & get the same hash to search for in my records. $\endgroup$
    – Sam
    Commented May 7, 2019 at 1:20
  • $\begingroup$ If someone stole my dataset, it'd be meaningless to them, as it's all hash, but, it's still functional to us if we get queries after our anonymisation process as we can lookup email addresses and check the none personal data fields that weren't hashed, like contact dates, number of attempts and outcomes etc.. $\endgroup$
    – Sam
    Commented May 7, 2019 at 1:23

1 Answer 1

Reset to default
1
$\begingroup$

Even with PBKDF2, You wouldn't be able to use salt anyways, because salting relies on a one-to-one mapping between the salt and a user identifier (email). You cannot keep the email, so you wouldn't know which salt to use when a user comes back with his email.

Simply iterating SHA 256 for a large number of times is a way to discourage preimage searching. Other (more solid) cost-incurring algorithms include scrypt. Although time-space balance attacks still work if you don't have salt to work with. (Using SHA, possibly inserting large constant data after input string to further slow it down).

Improve this answer
$\endgroup$
6
  • $\begingroup$ Note that iterating SHA-256 can slightly reduce security since it risks hitting a short cycle. This isn't a realistic concern for a good 256-bit hash function, but it'd be dangerous to conclude that any hash can be made slower more securely just by iterating it. $\endgroup$
    – forest
    Commented May 3, 2019 at 1:31
  • $\begingroup$ @forest, Hi, I'm confused to what you are saying. I'm thinking that if you have some input, and then you do SHA(x) = y, and then SHA(y) = z and so on, then you are in a SHA cycle. But however many iterations of SHA you do, you are still in this same cycle, is that right? $\endgroup$
    – nuts-n-bits
    Commented Dec 17, 2019 at 19:43
  • $\begingroup$ There are a large number of cycles, some large, some small. For a hash with a large output space, most are going to be very large, but there's always a chance that you can hit a small one. $\endgroup$
    – forest
    Commented Dec 20, 2019 at 21:58
  • $\begingroup$ Yes. But is it possible to enter a small cycle after reaching a large cycle first? $\endgroup$
    – nuts-n-bits
    Commented Dec 28, 2019 at 5:42
  • $\begingroup$ Not sure what you mean. You're only going to hit one cycle and cycles are, by definition, cyclic, so once in one, you're stuck. It's better to be stuck in one so huge that there are no security issues than one which is small enough that the space may be enumerable. $\endgroup$
    – forest
    Commented Dec 29, 2019 at 23:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.