It's not the algorithm that's more "efficient" (that's just a welcome side-effect) but the security level.
Security levels are usually given in bits; to say that a cipher has 80 bits of security means that we assume it takes roughly $2^{80}$ effort to break it, for some definitions of "effort" and "break".
For RSA, the main problem is factoring large numbers. Factoring algorithms are a lot better than brute force - factoring a $n$-bit RSA modulus takes a lot less than $2^n$ operations. Based on the speed of current factoring algorithms, the latest guess (according to ENISA) is that for 80 bit security your $N$ should have around 1024 bits length and for 128-bit security you need a 3072 bit $N$.
For libsodium, the main problem is taking discrete logarithms over a particular elliptic curve group of order $2^{255}-19$. With the best-known algorithms today, taking a discrete log over such a group (where the elements have about 256 bits length) would take around $2^{128}$ operations, hence you get 128 bit security with a lot smaller elements.