5
$\begingroup$

I have noticed that different implementations of SHA1 give different results. As I'm currently programming my own SHA1 implementation (in MATLAB / GNU Octave), I wonder which one should I follow?

There are several implementations of SHA1 in Rosetta Code. Using the message Rosetta Code, most of them give SHA1 hash of 48c98f7e5a6e736d790ab740dfc3f51a61abe2b5.

There is also Tools for Noobs Online sha1() function, that gives the same hash as the examples in Rosetta Code.

In Rosetta Code, there are also two examples of computing SHA1 in Unix shell:

In my own computer (Debian Linux Wheezy amd64),echo 'Rosetta Code' | sha1sum gives a SHA1 hash of d350d93ccda0fbf2d5037dedd2f3763fe880b2c5 and echo 'Rosetta Code' | openssl sha1 also gives the same hash.

However, the Rosetta Code examples of computing SHA1 in Unix Shell use different message: ASCII string . According to Rosetta Code, echo -n 'ASCII string' | sha1 should give 9e9aeefe5563845ec5c42c5630842048c0fc261b, and echo -n 'ASCII string' | openssl sha1 | sed 's/.*= //' should give the same hash. When I try ASCII string as message in Tools for Noobs Online sha1() function, it gives the same hash as the examples of Rosetta Code on computing SHA1 in Unix shell. So I can conclude that all these implementations of SHA1 in Rosetta Code and also Tools for Noobs Online sha1() function give identical hashes.

In my own computer, using the message ASCII string I get the hash 4f9e000b925f5d96def1d9b3e21cdc11f1df6139 either by echo 'ASCII string' | sha1sum or echo 'ASCII string' | openssl sha1.

So, of all these implementations, only my own computer's sha1sum and openssl sha1 differ from the examples of Rosetta Code and Tools for Noobs Online sha1() function. Has this something to do with the UTF-8 vs. ASCII, processor type, endianness, message suffixing differences or is there something wrong in my Linux configuration?

Improve this question
$\endgroup$
2
  • $\begingroup$ Sounds like something encoding related, but not UTF-8 vs. ASCII. Perhaps UTF-16? $\endgroup$
    – CodesInChaos
    Jul 13 '12 at 11:32
  • 9
    $\begingroup$ Your problem is that you left out the -n option of echo. Therefore echo appends a newline character to your string before the string is passed to sha1sum or openssl. $\endgroup$
    – Maeher
    Jul 13 '12 at 14:10
16
$\begingroup$

The echo command will include a newline at the end. So when you use echo 'Rosetta Code' | sha1sum you are actually hashing the string Rosetta Code\n.

Do the test using echo -n, the -n flag prevents the trailing newline character. Doing echo -n 'Rosetta Code' | sha1sum gives the same 48c98f7e5a6e736d790ab740dfc3f51a61abe2b5 hash that you were seeing elsewhere.

Improve this answer
$\endgroup$
3
  • $\begingroup$ you are right!, but what should be done with the condition where the string already has new-line at the end of it example 'Rosetta Code\n' echo -n 'Rosetta Code\n' | sha1sum Seems like when we use 'echo -n' it removes all the trailing new-lines, including the one we have. I am asking it because, I am stuck with calculating sha1sum with above condition. $\endgroup$
    – Mahesh Pujari
    Apr 2 '15 at 7:20
  • 1
    $\begingroup$ @MaheshPujari This is not a crypto question – maybe ask it on Unix Stack Exchange. (For me, echo -n doesn't seem to remove any newline, it just doesn't append a new one.) $\endgroup$
    – Paŭlo Ebermann
    Apr 4 '15 at 11:38
  • $\begingroup$ @MaheshPujari Remember that echo doesn't interpret escapes, without the -e flag. So `echo -n 'Rosetta Code\n' is outputting the literal string 'Rosetta Code\n', with no new lines at all. $\endgroup$
    – moveaway00
    Sep 8 '15 at 5:01
4
$\begingroup$

SHA1 is fully described (including complete test vectors) in this FIPS publication, if you wanted to do it properly. But I think checking against the final digests of a couple inputs should be enough to point you to the correct version. I extracted them for you (inputs in ASCII, quotation marks excluded):

"abc" (3 bytes)

A9993E364706816ABA3E25717850C26C9CD0D89D

"abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq" (56 bytes)

84983E441C3BD26EBAAE4AA1F95129E5E54670F1

If you don't get those with any of your tools (endianness aside), well, to quote NIST, you may have a problem.

If you are implementing SHA1, be sure to keep the .pdf I linked close to hand, it will be invaluable - it provides test vectors for every round and for many inputs, and can be used for SHA2 as well.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.