Can counter mode be used in some way in public key cryptography? Can we decrypt when counter mode is used if 1. we have the key and 2. we have the cipher text? Is there any other mode in which this is possible?
I interpret your question to be asking "Can we directly use public key encryption algorithms with a CTR-like mode?", as opposed to "is it possible to perform a key exchange via RSA and use AES-CTR?" (which is obviously possible).
Let's define a "block cipher" as a deterministic algorithm that produces a permutation (the "cipher" part) on a fixed-length block of bits (the "block" part). Non-randomized RSA meets this definition, though there are technically variants with different block sizes, determined by the modulus size.
The question then becomes "Can we apply textbook RSA to a public stream of incrementing numbers?", which is obviously a yes. However, doing so will not be secure. Non-randomized RSA is understood to be insecure. More importantly here, you will also be sending the nonce, and the adversary is assumed to possess your public key. Thus, nothing will prevent an adversary from applying $E(seed, public\ key)$ and generating the key stream himself.
"Randomizing" the above construction by using a random nonce will not work - if you are sending the nonce alongside the ciphertext, and the adversary has the key, the adversary can trivially recover the message.
So a secure CTR-mode using deterministic public key encryption such as RSA appears to be doomed.
Can we decrypt when counter mode is used if 1. we have the key and 2. we have the cipher text?
No, CTR mode generates a key stream by encrypting a counter, and adds the key stream to the message. Thus, if you do not have the seed for the key stream, you will not be able to regenerate the key stream and recover the message (assuming you do not resort to cryptanalysis to recover the message, anyways). While not sending the seed will disallow an adversary from recovering the message, it will also disallow the intended recipient from recovering the message, as both of them possess identical keys.
Public key cryptography and counter mode do not directly interact; it simply does not make sense.
Public key cryptography is focused on providing a way to encrypt something with a (public) key, which can only be decrypted with a (private) key. This is called an asymmetric cipher. Asymmetric ciphers are computationally expensive to use due to the large numbers involved.
Now CTR or counter mode is a way to chain together blocks encrypted with a so-called block cipher. Block ciphers (like AES or Serpent, for example) only produce a good encryption for a very small size of data, a block. They are usually heavily optimized for working with today's computers and are perfect to use with large amounts of data, though. But they work symmetrically. Which means, you can decrypt any block with the "passkey" it was encrypted with. To overcome this limitation, the use of a Block-Chaining-Mode is necessary, which varies the used "passkey" for each block.
But to answer your original question: yes, it is done. But only indirectly and called "hybrid encrpytion". Have a look at OpenPGP for that.
The way this works is that the actual initial passkey is encrypted through asymmetric encryption, and only by decrypting that you get the symmetric key (also called session key) required for the actual message. It combines the speed of symmetric encryption with the security of asymmetric encryption.