What you know from the public key:
What you need to calculate $d$:
The functions $\phi$ and $\lambda$ are multiplicative functions, which is essential in the following way: The calculation is straight forward, if you know the factorization of $n$. In general the way to calculate the functions for composite values is to factorize them first.
In the original paper A Method for Obtaining Digital
Signatures and Public-Key Cryptosystems by Rivest, Shamir and Adleman (1978), they already noted a probabilistic algorithm to calculate $\phi(n)$ if you know $e \cdot d$ (which can be quite a lot larger), which is basically Miller's algorithm described in Riemann's hypothesis and tests for primality, Miller (1975). In 2004, May published Computing the RSA Secret Key is Deterministic Polynomial Time Equivalent to Factoring, which is a deterministic algorithm for that problem, showing that factoring and calculating $d$ from $e$ and $n$ is equivalent in polynomial time.