Say I want to verify the identity of Alice, but Alice could be colluding with Bob to fool me. Is there any way to verify Alice's identity and also be sure that Bob is not impersonating Alice, e.g. using her private key?
No, this is not a security feature provided by asymmetric signature schemes, the way the concept is usually understood.
You have Alice's public key. Typically, you got it in a certificate issued by Carol. Let's assume (for the sake of argument) that Carol has a very strict certificate issuance policy, and only issues certificates corresponding to hardware devices with 100% reliable biometric identification mechanisms. Hence, when you receive a signature that verifies OK with Alice's public key, you can be sure Alice was present when the signature was generated. However, the signature doesn't tell you whether Alice was on the phone with Bob when the signature was generated.
If you want a signature to proves that the signer had no out of band communication with anyone at the time of the signature generation, you must make sure that the private key is stored and used in such way that no such communication would have been physically possible. Say, Carol has a policy only to issue certificates of the kind you require for hardware that consists in a remotely monitored Faraday cage that only fits a single person.