The answer is different for the schemes that are used. So there is no one answer for the simple reason that the different modes have different properties.
In general modes such as PKCS#1 v1.5 or OAEP encryption is used for RSA. These modes contain a specific padding that is performed before the modular exponentiation that is the backbone of RSA. Performing the modular exponentiation with the private key - possibly sped up using the Chinese Remainder Theorem or multi-prime calculations - will result in the padded message. Now the padding is checked before the message is returned. This means that using an invalid private key / ciphertext pair will result in a padding error. Note that padding errors may actually lead to attacks on the key.
It is not possible to perform encryption directly using ECC. Instead ECIES can be used to perform it indirectly. This is basically key agreement followed by encryption using a symmetric key. In this case the asymmetric key agreement may fail without error resulting in an incorrect symmetric key. Then the subsequent decryption may not cause an error either. This can be fixed by using an authenticated cipher such as GCM, which generates an error for any invalid secret key / ciphertext pair.
The same goes for RSA-KEM as for ECIES. So again: it depends on the scheme.
So the conclusion: use an authenticated encryption scheme for the symmetric cipher. And don't forget to protect against padding oracle attacks for RSA. PKCS#1 v1.5 padding should not be used anymore because of a specific padding oracle attacks called the Bleichenbacher attack. OAEP should be secure (if the implementation is secure, of course).
If you do that you can be sure that some kind of error is returned if you use an invalid private key / ciphertext pair.
Another option is to sign the plaintext message. In that case you can verify the authenticity of the message after decryption. This has the additional advantage that an adversary cannot simply encrypt a different message using the public key. In general this should be combined with the authenticated encryption option mentioned in the previous section.