Given that hash functions are inherently irreversible, would they not then be useful in applications where it's desired to overwrite sensitive data? (i.e. Darik’s Boot and Nuke)

Specifically, why should hash functions not be used for data clearing?


Specifically, why should hash functions not be used for data clearing?

I take it you mean that we would overwrite the storage with output produced by feeding its sensitive contents to a hash function. The problem then is that this output, being a function of the data you're overwriting, therefore provides information about that input.

In particular, it could be used to efficiently test hypotheses about the content of the overwritten data. This is the same principle as password cracking; just like password hashes allow you to quickly test password guesses, the overwritten disk sectors' contents could perhaps be used to test whether the disk contained or not some specific data before being overwritten.

You may think that this sounds like a very unlikely attack, and you could well be right about that. But why allow it in the first place? Particularly when the alternative is simple enough: overwrite with random or pseudorandom data that has no correlation at all to the sensitive contents you're overwriting.

  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – e-sushi Dec 29 '16 at 1:39

Specifically, why should hash functions not be used for data clearing?

Several reasons, the main being that there is no need to use one.

If you were thinking about reading the data from the device in chunks, running it through a hash function of equal size, Luis' answer points out why that is a bad idea from a security perspective. It is also slow, as it requires reading data from the disk as well as running the hash function. Data overwrite operations should not be reading from the disk, except to verify that data was written correctly.

It is also not a good idea from a technical perspective on magnetic storage. Wipe patterns are designed to normalize the magnetic strength of each bit of data, so there is not a "ghost", which may read as all 0s, yet still indicate the presence of prior data if read and interpreted with special hardware or drive firmware. There are specific wipe patterns designed to eliminate these depending on the magnetic encoding used to store the data on the media, the areal density, and the method of writing to the platters.

Any reallocated sectors will also be unaffected by a software wipe, as will any restricted sectors that the operating system denies access.

Modern storage devices

Modern drives have specific commands to properly erase a disk without sending write commands to every sector. The Enhanced Secure Erase* command will wipe all sectors with a single pass, including reallocated and OS restricted sectors. It is also significantly faster, an erasure that would take hours using software may take only minutes, as the commands are directed by the drive controller instead of the operating system. Enhanced Secure Erase* and Sanitize Overwrite* are equivalent on magnetic drives.

Encrypted and some unencrypted drives may also support Crypto Erase, which deletes the encryption key and overwrites it with random data, taking only a few thousands of a second to complete. This does not actually change the data on the storage media, only the controller, but is generally regarded as secure if there is no backup of the key. Unencrypted drives supporting this command do encrypt the data before writing it, but do not expose this is a security feature except for the Crypto Erase functionality.

SSDs support a Block Erase command instead of the Sanitize Overwrite command, which clears the NAND cells of charge directly instead of writing data to them. This is faster and more secure than writing 0s, either through the controller or through software. Using the Overwrite command designed for magnetic media is generally not effective at erasing SSDs and should be avoided.

Combining the Crypto Erase commands with either the Enhanced Secure Erase, Sanitize Overwrite, or Block Erase commands should render a drive's data completely unreadable, even by the most sophisticated of recovery techniques. For the extra paranoid, finish the task by filling the drive with pseudorandom data.

Of course there are other devices, like SD cards and USB flash memory sticks that do not support specific erasure commands, so they need to be erased with a software overwrite. A hash function will not provide a wipe pattern that is effective for these devices. Generally you need a minimum 3 pass wipe of alternating patterns before data recovery is infeasible. NSA device declassification protocol is device destruction or high strength degaussing. HAMR drives may not be completely erased by degaussing.

Older storage technologies

If you are dealing with magnetic media that does not support any kind of ATA erase commands, it is generally accepted that a single pass wipe of all 0s, similar to the internals of the Secure Erase command, is effective against all but an electron microscope. Extremely old devices using modified frequency modulation with low areal densities may require multiple wipes with specific patterns.

I am sure you do not have one of these, think sub 100MB hard disks, but if you did, just destroy it. The advantage of a software wipe is that you can use the drive after, but nobody needs or wants the kind of hardware that actually requires multipass wipes, and the cost of electricity to perform a 35-pass wipe on some old drive is less than the cost of throwing it in an incinerator or filling the drive with hydrofluoric acid.

*not all drives properly support ATA erasure commands, check with the drive manufacturer before trusting that these are effective, and always verify they completed without error

  • $\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$ – e-sushi Dec 29 '16 at 4:31

You could argue that they are. If you use a utility like shred it will over write a disc with pseudo random data which might be generated via a cryptographically secure random number generator. These are often based around hash functions, using some sort of counter loop. Boot and Nuke can use the PRNG ISAAC which is a random number generator based on hashing it's output.

You realise that this topic is a can of worms located in the middle of a mine field? It is very very difficult to destroy data. There are no known 100% reliable means of soft data erasure for SSDs or flash drives. More details can be found by addressing the good folks at security.SE.

  • $\begingroup$ Your answer is correct, but would you agree that a hash provides no advantage over other similar approaches? $\endgroup$ – bmm6o Dec 15 '16 at 0:08
  • $\begingroup$ @bmm6o some would say that simply hashing disc sectors in groups of say 20 bytes (SHA1) would be counter productive. Most over write erasure patterns require just that, patterns. All zeros, all ones and random numbers e.t.c. A hash only approach would skip the former and weaken the erasure. And where would the secure random numbers for over write originate? $\endgroup$ – Paul Uszak Dec 15 '16 at 0:53
  • 1
    $\begingroup$ You might want to correct your statement about SSDs as there are known ways to securely erase SSDs drives. If you look at the website of Boot and Nuke which you mentioned yourself, you'll notice the business version of DBAN called Blancco is one of them. Blancco offers secure SSD data deletion. Quoting the DBAN.org website: Should you need to erase data from a SSD or require a certificate of data removal, request a free trial of Blancco Drive Eraser. $\endgroup$ – e-sushi Dec 29 '16 at 1:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.