Given a modulus $N$ and a number $a$, a multiplicative inverse exists for $a$ if $a$ and $N$ are coprime. Why isn’t there a cryptosystem that uses this as a computational problem?
Alice and Bob agree on a public modulus $N$ and a public number $p$ such that $p$ and $N$ are not coprime. Then Alice sends Bob $a \cdot p \bmod N$ and Bob sends Alice $b \cdot p \bmod N$. Where $a$ and $b$ can be any number. Then they compute the shared secret: $a (b \cdot p) \bmod N \equiv b (a\cdot p) \bmod N$.