While it is true that Elliptic Curve Diffie Hellman, Elliptic Curve Signature Generation and Elliptic Curve Signature Verification rely on scalar multiplications, these are usually implemented as different types of scalar multiplication for both security and efficiency reasons.
In fact there are three types of scalar multiplications used in practice for these schemes:
- Fixed-Base: when the input point of the scalar multiplication is known at design time
- Variable-Base: when the input point of the scalar multiplication is not known in advance
- Double-Base: when the protocol actually requires to compute two scalar multiplications and then to add the both results. (e.g. $kP+rB$ )
Variable-Base is what is usually needed in ECDH, as you receive the input point of the scalar multiplication from another peer.
Fixed-Base is used in key generation and signature generation. Here the fixed-base is always the "generator point", which is the point who generates the prime-order subgroup that is provided together with the curve's equation.
Double-Base is used for signature verification (both ECDSA and EdDSA and Schnorr) require to compute something of the form $kP+rB$. Note that unlike ECDH or Signature Generation the Signature Verification doesn't make use of any secret values and therefore there are no requirements for constant time execution.
X25519 provides a very simple, constant time, and fast variable-base scalar multiplication algorithms. This is very good for ECDH and this is why it is used specifically for ECDH.
Ed25519 instead provides a very fast fixed-base and double-base scalar multiplications, thanks to the fast and complete twisted Edwards addition law.
In fact, the fixed-base algorithm of Ed25519 is, on most platform, faster than the variable-base of X25519.
And using X25519 for signature verification is just a bad idea because you need to execute it twice (to compute $kP$ and $rB$) and then to add a new function to perform the full addition of the two resulting points. Furthermore you are using constant-time computation where it's not needed. Instead, you can achieve a much faster implementations using interleaved NAF methods and the twisted Edwards addition law.
Note: for the purpose of this answer I'm not distinguishing between different elliptic curve signature algorithms such as ECDSA and EdDSA.