No, that doesn't work.
If one party chooses primes $p,q$ and sets $n = pq$, then other parties would also have to know $p$ and $q$, because it is the only way to get the same $n$.
But you just left out a part of the public key, which is $g$. This results in a different question:
If you have a trusted party set up $n$ and assign different $g$ values to each party, would that work? No it would not, because in the decryption process you need to calculate $c^{\lambda}$. And knowing $\lambda$ would allow the factorization of $n$. So basically every party would be able to decrypt the messages to every party, as if they were just using one public key.