This is indeed possible, however there is a major disadvantage to doing so, namely that RSA key generation is slow.
You have to generate two large prime numbers, which is done by picking random odd numbers and checking whether they are prime. The probability of a number $x$ being prime is around $1/\log_e(x)$. This means that the probability of an $n$-bit number being prime is around $1/0.3n$. Finding two $1024$-bit primes will take a few hundred goes, which takes a nontrivial length of time.
Diffie-Hellman—whether in a bog-standard prime field $\mathbb{F}_p$ or over some fancy elliptic curve $\mathbb{E}(\;\cdot\;)$—uses uniformly distributed group elements, which are only slightly more difficult to find than a random string from a PRNG, making this option much more efficient than generating an ephemeral RSA key.
If quantum computers turn up, then the approach that you discuss is the most straightforward way to go about replacing Diffie-Hellman in the TLS handshake, as the algorithms that I know about provide encryption and—occasionally—signatures, but nothing like Diffie-Hellman.