If you use the raw RSA operation ($M^d \bmod n$ or $M^e \bmod n$), then no, it is unsafe to use the same key, because an attacker could trick the private key holder into signing a message $M$ (i.e. generating $M^d$) which is actually an encrypted message ($M = P^e$), thus allowing the attacker to recover the original plaintext ($(P^e)^d = P$). (The dual attack leads to a forgery.) However, the raw RSA operation has a whole range of other flaws that make it unsafe as an encryption or signature scheme.

Encryption and signature schemes based on RSA use padding modes. The standard padding modes include discriminants so that an encryption payload does not look like a signature payload. Therefore, it is safe, cryptographically speaking, to use the same RSA key pair for signature and encryption, provided that the key pair is used safely for signature and used safely for encryption.

However this is a bad idea for a different reason: key management. Signature keys and encryption keys have different requirements in terms of backups, access control, repudiation, etc. The fallback for a signature key in case of a catastrophic event is to destroy it to avoid future forgeries, so a signature key does not need to be backed up extensively. Conversely, the fallback for an encryption key is to keep it around to decrypt existing documents, so it needs to be backed up reliably. In case of a leak of a signature key, pre-existing documents are not affected; it is only newly produced signatures that must be repudiated. Conversely, in case of a leak of an encryption key, the confidentiality of all pre-existing documents is at risk, whereas new documents simply need to be encrypted with different keys. The constraints push in different directions, so the keys need to be managed differently, they need to be different.

The short answer is no. This is a general piece of wisdom in cryptography: never use the same key for more than one thing. A “thing” means a specific scheme where all the parameters are fixed apart from the key itself and the message size. Don't use the same key to encrypt and sign; don't use the same key with both PKCS#1v1.5 and PSS; don't use the same key with different hash functions.

If you use the raw RSA operation ($M^d \bmod n$ or $M^e \bmod n$), then no, it is unsafe to use the same key, because an attacker could trick the private key holder into signing a message $M$ (i.e. generating $M^d$) which is actually an encrypted message ($M = P^e$), thus allowing the attacker to recover the original plaintext ($(P^e)^d = P$). (The dual attack leads to a forgery.) However, the raw RSA operation has a whole range of other flaws that make it unsafe as an encryption or signature scheme.

Encryption and signature schemes based on RSA use padding modes. The standard padding modes include discriminants so that an encryption payload does not look like a signature payload. Therefore, it is technically safe, cryptographically speaking, to use the same RSA key pair for signature and encryption, provided that the key pair is used safely for signature and used safely for encryption. Here, I mean safe in a narrow sense, assuming a “perfect” world where everything that should stay secret, remains secret.

The thing is, the world is not perfect. Implementations of RSA can leak partial information through side channels. Keys get compromised. Protocols using RSA sometimes use it in ways that are very brittle.

Using the same key for both encryption and signature can exacerbates weaknesses. RSA has a history of weaknesses, not in the mathematical algorithm itself (once you use proper padding), but in the way it's implemented. Decryption, especially, requires a lot of care, because some inputs are invalid, and the mere act of revealing whether an input is valid or not can allow oracle attacks, the best known of which is due to Bleichenbacher. The way RSA decryption is used in TLS¹ exacerbates this because it creates a second reason for invalidity: a ciphertext can be invalid for the decryption operation, or can be valid for decryption but decrypt to something invalid. This has led to a long series of practical attacks against TLS cipher suites that use RSA decryption. Using the same key for both encryption and signature has two downsides. Most obviously, if a vulnerability in decryption allows the attacker to recover the key, then anything that relies on a signature is also broken. Less obviously, even if the attacker can't recover the key, but can get some partial information about the decryption of invalid ciphertexts, they may be able to forge signatures. This is the case, for example, with the CAT variant of Bleichenbacher's attack, so that even clients that do not accept RSA decryption ciphersuites are still at risk if the server uses the same key for RSA signature ciphersuites.

Another reason is key management. Signature keys and encryption keys have different requirements in terms of backups, access control, repudiation, etc. The fallback for a signature key in case of a catastrophic event is to destroy it to avoid future forgeries, so a signature key does not need to be backed up extensively. Conversely, the fallback for an encryption key is to keep it around to decrypt existing documents, so it needs to be backed up reliably. In case of a leak of a signature key, pre-existing documents are not affected; it is only newly produced signatures that must be repudiated. Conversely, in case of a leak of an encryption key, the confidentiality of all pre-existing documents is at risk, whereas new documents simply need to be encrypted with different keys. The constraints push in different directions, so the keys need to be managed differently, they need to be different.

¹ _{That is, ciphersuites whose name starts with TLS_RSA_. Ciphersuites whose name starts with TLS_(EC)DH(E)_RSA use RSA signature, not RSA decryption, and are a lot more robust.}