# Is there a standard method to compare hardware-implemented ciphers?

I have been reviewing literature and I am trying to determine the best method to compare ciphers in hardware. The two methods that seem to be somewhat de facto standard the literature are looking at throughput for a 100kHz clock, and looking at area in GE (gate equivalents). I take issue with these methods because for the clock method, the initial state is not given, ie: I do not know if this is measured from a loaded state. The second method I have an issue with because GE is the area of a NAND gate, and if you use a huge NAND, you look like your implementation is relatively better.

Is there a standardized method to compare ciphers on a hardware level that is not this seemingly de facto method?

Furthermore, is there a standard method to compare Feistel ciphers against a substitution-permutation network?

If someone could point me to a document that outlines a method to compare ciphers from the implementation standpoint, it would be helpful.

• "Relatively better" may be too open of a comparison - Better in what situation? For example, is a cipher with lower latency better then a cipher with higher throughput? Similarly, is a cipher with a lower latency but larger area better then a cipher with a higher latency but lower area? I think you would need to define the specific use case before you can make comparisons - the ideal cipher for one situation is probably not ideal for other situations. You might check out how AES was selected, though I don't know if it was a "standardized" method – Ella Rose Feb 13 '17 at 18:02
• @EllaRose I realize that this is a loaded question, and you hit some of the major points. I'm ignoring the "software implementation" side of things, and I hope to make a grid that has throughput as "bytes/watt". The "bytes/watt" item is useful only to me and a few others in the power-constrained space, but I thought I'd start by implementing the ciphers of interest in a "standard" way, and then doing the bytes/watt comparison. There just seems to be no standard, for the issues that you mention, but one can hope that there's some document out there that I am yet to find. – b degnan Feb 13 '17 at 18:18
• IME, the "standard way" to implement a cipher for benchmarks is either using the best publicly available existing implementation, or the best implementation you can reasonably produce yourself. While not 100% objective, that's a practically relevant metric, since that's more or less how a typical user would implement the cipher if they were using it in production. – Ilmari Karonen Feb 13 '17 at 18:45
• ... Just remember to document the implementation choices you make reasonably carefully, so that readers can tell if you're e.g. comparing a highly optimized implementation of cipher A with a naïve reference implementation of cipher B, and if so, why (e.g. there was no optimized implementation of cipher B available for your platform). – Ilmari Karonen Feb 13 '17 at 18:45
• @bdegnan I think the above comment is more or less the practical answer, but technically the question asks if there is a "standardized method", which I interpreted to mean some sort of officially proscribed guideline (i.e. FIPS), as opposed to what people usually do in general (i.e. the de facto standard method). Was that your intention? – Ella Rose Feb 13 '17 at 19:47

Because of how many ways there are to implement a hardware design there is not a specific comparison protocol that I am aware of. The main comparison points are power, area, and throughput, and comparisons are generally done on the same process, 32nm GF SOI for example. With a given process, area can be measured in GE or mm$^2$.

The algorithm itself will usually be the limiting factor when it comes to clock speed, and this may not be an issue if the throughput is still good. Unrolling loops uses more area but can increase throughput dramatically. Constants like those in SHA2 use a large amount of area, wheras those in SHA-3 use very little.

An implementation can be tuned to be better at a specific trait, such as power usage or speed, or tuned to have a good combination of two of them at the expense of another. Throughput/area or throughput/watt then become the usual target for a given implementation.

    Area          Power         Throughput
A   1mm^2         10mw          1.00Mb/s
B   1mm^2         30mw          2.45Mb/s

C   3mm^2         10mw          2.45Mb/s
D   3mm^2         30mw          6.00Mb/s
E   3mm^2         90mw          14.7Mb/s

F   6mm^2         90mw          44.1Mb/s


Each implementation has an advantage and a cost. D has a 100% increase in throughput/area and throughput/watt over A, but at triple the area. B and C have the same throughput, but one uses less area and the other less power. For a given area, triple the power budget gives a 2.45X increase in throughput. Because the 3mm area implementation is able to radiate more heat thanks to increased surface area, it can be pushed beyond the power limits of the 1mm area implementation. Despite using 1/3 the area and 1/9 the power, A is only 14.7 times slower than E. F is 3X faster than E at the same power, with only double the area used.

So how do you compare them? Which one is the best? The high area implementation obviously has the best throughput, and the best throughput/watt, but you may not have the area to spare, or the power. Would one consider A more efficient than E or the other way around? E has 63% better throughput/watt than A, but at the same wattage A has 22% better throughput/area.

In the end the a good comparison can only be made by keeping the process a constant, then deciding which implementation is better or more efficient based on the requirements of the final product.

• You described what I was planning on doing, so it's good to have a consensus, but I was hoping there was a standard somewhere. I have a pretty good framework to compare things, so I'll write a comparison document while I compare the implementations. btw, due to metal rules, at 14nm and 32SOI, the areas are about the same. It's nuts down there in FinFET land. – b degnan Feb 14 '17 at 12:58
• I have written a "lightweight AES" reference implementation that I am planning on putting down onto 14nm. I was wondering if you would review it. If you are interested, please find me and get in touch. – b degnan Jul 12 '17 at 15:27
• I've documented my S-Box implementation for AES, and the tool chain that I will be using for comparisons to Simon here: github.com/bpdegnan/aes – b degnan Mar 28 '18 at 12:16