Computations on elliptic curves are more efficient. Roughly speaking, when the base field has size $n$ (for DH/ElGamal/DSA, the size in bits of the modulus $p$; for elliptic curves, the size of the field for point coordinates) and a "security level" $t$ (e.g. $t = 80$ for "80-bit security" as can be expected when using a 160-bit subgroup and a 160-bit hash function), the computational cost of a private key operation (decryption, signature generation) is roughly $O(n^2t)$, both for the modular and the elliptic curve variants. The big-O notation hides a constant, which is about $10$: for the same $n$ and $t$, the elliptic curve variant will be 10 times slower than the older modular algorithm.
However, elliptic curves tolerate much lower $n$ for a given security level. Basically, for $t = 112$, we need $n = 2048$ for the modular algorithms, but $n = 224$ is sufficient for elliptic curves (these comparisons are always a bit subjective, see this site for details). Report the values in the formulas: even with the factor 10, the elliptic curve variant will be more than 8 times faster than the modular algorithm of comparable strength.
Another point is binary fields. When the implementation platform is a dedicated ASIC/FPGA, or even when the platform is a small CPU which does not offer a very optimized multiplication opcode (e.g. the ARM Cortex M0), then curves on binary fields give a huge implementation boost, especially Koblitz curves. It also has benefits for memory-constrained systems, see for instance this presentation. The "modular" algorithms cannot really benefit from binary field computations (it can be defined, but discrete logarithm on binary fields is easier than modulo a random prime, so a binary field variant of modular DH or ElGamal would require a $n$ twice larger, hence a four-fold slowdown).