End-to-end encryption refers only to the -- somewhat tautological -- fact that encryption happens on both ends and any intermediates do not have access to the keys, not to the specific math behind it.
Since the keys have to be available to both ends, public key cryptography is usually the preferred method, because the private key for each end is not transmitted to the other end, and therefore cannot be intercepted unless either end is compromised.
Also wether it is factoring-based (i.e. RSA algorithm), or elliptic-curve based refers to the method of getting the function leading to the encrypted message, which is difficult to reverse.
The basis of cryptography is quite algebra-heavy, depending on your background; that said one introductory book is Hoffstein's: springer.com/us/book/9781493917105
Either way it's probably easier than going through the original literature on the topic. Otherwise, you may want to start with a more conceptual primer, here's one
hope that helped.