How to count Gate Equivalent (GE) of Sbox used in block Cipher. For example how much area does the Sbox of Block Cipher Prince take? And How is it calculated?
This basically boils down to find a "good" implementation of the S-box and then compute the GE by weighting the operations used (e.g. XOR = 1GE), that is, as you already commented, synthesize the implementation.
Note that the resulting GE greatly depend on the optimization of your implementation, the technology library used etc.
Wikipedia lists some software for this:
I've never used a tool to design an ASIC, I have only some basic experience with FPGA development, so cannot tell you more about the different tools.
"Gate equivalence" is just a marketing tool, and has no real relevance for serious work. The only thing that matters in the end is whether you have used up the actual resources (LUTs, FFs, RAMs, routing resources, etc.) of any particular chip with your actual design. If you give me "GE" in a paper, I will most likely eat you alive on the reviews as you are most likely peddling snake oil.
In order to compare S-Boxes, you need to make the S-Boxes in a target process. As in example, I had to make a reference AES S-Box to do power comparisons against other ciphers. I then take those circuits and put them against layout at different nodes.
The standard EDA (Electronic Design Automation) tools for hardware synthesis take as input a design and a technology library and produce a gate level equivalent design with the technology library (e.g. CMOS 45nm, FiNFET 14nm, FDSOI 28nm, in the case of PRINCE paper, the authors used Nandgate 45nm and UMC 90nm)
Additional output describes some properties of the gate level output, including the area expressed in e.g. squared millimeters.
The technology library is basically a list of gates and describes how they are made, and in its documentation you can find how big they are (again, in area).
The way to compute GE is to divide the area of your synthetized, gate level design by the area of a NAND2 (that is a NAND with only 2 input) of your target technology library. This figure is supposed to be approximately independent on the target technology, so you can use it to make estimation of the area of the same design in different technologies.
In practice the approximation error is about 5% or 10% but it's still a good estimation and widely used.
In papers, when the design is small, e.g. an SBOX, you can often count gates manually and then compare it with your technology NAND2 gate. See, as example, section 5.4 of this very good paper on AES SBOX.