This is, by definition, impossible using public-key encryption alone: the public key is assumed to be public, so that everybody knows (or can, if they wish, learn) it and can encrypt messages using it. If you change that assumption, what you have is no longer public-key encryption but something else with different security assumptions and definitions.
That said, you can get something like what you're asking for by combining public-key encryption and digital signatures. Basically, let's assume that Alice wants to allow Bob and Charlie to send her encrypted messages, such that:
Only Alice can read the messages.
Only Bob and Charlie can generate messages that Alice will accept as valid.
Even if Bob's and/or Charlie's full key material is leaked, others will not be able to read the messages they've sent (or will send) using those keys.
Even if Alice's key material is leaked, others will not be able to use it to generate valid messages that Alice would accept.
To achieve this, Alice can:
Generate an encryption key pair and a signature key pair.
Keep the private (i.e. decryption) half of the encryption key pair, and the public (i.e. verification) half of the public key pair.
Send the other halves of the keys (i.e. the public encryption key and the private signing key) to Bob and Charlie, and erase them (or at least the signature key; there's no need to erase public keys, since they're public) locally.
(Alternatively, Bob and Charlie can each generate their own signature key pairs, and send the public verification keys to Alice. This avoid having to transmit private keys between parties and erase them, and also prevents Bob and Charlie from impersonating each other.)
Now, to send a message, Bob (or Charlie) should encrypt it using Alice's public key, and sign it using the signature key,* and send both the encrypted message and the signature to Alice. Alice can then decrypt the message using her private decryption key, and verify the signature to prove that it came from Bob or Charlie.
*) It may or may not matter whether Bob first encrypts the message and then signs the ciphertext, or first signs the message and then encrypts the message and the signature. See this question for more details.