The speed of a cipher actually depends on lots of factors, including:
- The specific hardware platform you're considering (CPU architecture, instruction set, number of cores etc).
- Implementation details.
- Compiler flags used.
- Some ciphers have a large initial overhead due e.g. to a slow key setup; as a result they are slow when encoding very small messages.
Is this relevant to you? Or you're only interested in the asymptotic speed for large messages?
There is a large 2008 analysis of block cipher speeds in crypto libraries by one Timo Bingmann
Jump to the table in section 5.1.1 for a summary. You'll see that there is quite a lot of variation across libraries in relative speed. Blowfish overall could be considered the fastest cipher, but uses a 64 bit block and this disqualifies it from the comparison. Among 128-bit ciphers AES has the fastest implementation (40.1 MB/s in the Tomcrypt library) followed by Twofish (35.5 MB/s also in Tomcrypt). Serpent is usually slow (~15MB/s) but has a very fast implementation in the Botan library (30.2 MB/s) using bit-slicing.
For a more recent comparison you can also check this master's thesis `Block Ciphers: Fast Implementations on x86-64' by Jussi Kivilinna
This guy produced assembly implementations of a few block ciphers.
On an Intel sandy-bridge i5-2450M CPU these are the speeds on his fastest implementations in cycles/byte (lower is faster):
AES 1.35 (uses AES-NI);
AES 5.83 (no AES-NI);
Camellia 5.32 (uses AES-NI);
Camellia 14.10 (no AES-NI);
Note that there are other factors besides raw speed when considering implementations of ciphers (resistance to side-channel attacks and probably a million of other things).