5
$\begingroup$

I have implemented a bit-sliced version of two ciphers (Gibbon-120 and Hanuman-120, both part of the Primates family of ciphers: http://primates.ae/wp-content/uploads/primatesv1.02.pdf )

I now want to measure the increase in effectiveness that my bit-sliced ciphers have given me, compared to the reference implementations. Of course I also want to know my implementations performance/speed in a general context.

What I have tried so far, is to encrypt & decrypt 100byte, 100kbyte and 100mbyte 10 times, and then see what the average wall-clock time for encrypting and decrypting the different amount of data was. I doubt whether this is the smart way to go about it though. While a bit-sliced implementation is constant in execution time, the reference implementations are another matter. They use lookup-tables and thus can gain in speed after the first encryption, due to cachehits from lookup-tables now being a possibility. There is also the issue that if it is measured in wall-clock time, then background-processes on my computer could degrade or otherwise cause noise on the results.

On the other hand, I could count instructions in the compiled code (although I do not know how yet). This does not seem like the ideal way to go about it either though, as different instructions has different latency and throughput. If one cipher uses 10 additions and another uses 10 multiplication, then the former might be faster, despite them using the same amount of instructions. This is the case for some of the AVX instructions I used in my bit-sliced implementation (a permute instruction if I remember correctly takes 3x the time a shuffle instruction takes).

What is the standard way to go about this? What is the defacto standard that cipher-performance is being measured in, when a new up to date cryptographic cipher-design or implementation is being published?

My implementations are written in C for whatever it is worth (and uses Intel Intrinsics for AVX).

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Usually speed is measured as cycles per byte on long, medium and / or short messages. Sometimes medium and short messages are also omitted. As for the methodology, you may want to do some repetitions (of the program) and check whether they deliver the same result. $\endgroup$
    – SEJPM
    Commented Jun 5, 2016 at 20:58
  • 2
    $\begingroup$ You can check the NORX benchmark code; it loosely follows how SUPERCOP does it. $\endgroup$
    – Samuel Neves
    Commented Jun 6, 2016 at 0:50
  • $\begingroup$ Obviously there is no such thing as being perfect or ideal with regards to measuring performance ... $\endgroup$
    – Maarten Bodewes ♦
    Commented Jun 6, 2016 at 22:08
  • $\begingroup$ Thanks a lot for the feedback everyone! It was very useful, and I ended up going with cycles per byte with Intel turboboost off, hyperthreading off and running on only one core. The NORX Benchmark code helped!! :) $\endgroup$
    – oPolo
    Commented Jun 24, 2016 at 12:57

1 Answer 1

Reset to default
1
$\begingroup$

It is unfeasible with newer CPUs to just count instructions, as newer CPUs can do multiple per cycle, and have multiple cores and likewise.

Cycles per byte is the usual way to measure performance these days. Of course, to make it as fair as possible and comparable to others, the code is only ideally run on one core, turboboost and hyperthreading is turned off, etc.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.