Technically you're right, if you get the two prime-factors of n (p and q), then you can calculate everything else pretty easily.
But that's the problem! There is no known way to calculate p and q efficiently, but it is also unproven, if there really isn't a suitable algorithm and RSA is purposefully implemented in that way.
This is actually one of the big unsolved questions in mathematics.
Also your idea with storing a database of possible n's is infeasible, because this database would be unimaginably huge. You would have to store all possible calculations which would easily be greater than the number of atoms in the universe by far.