After generating a key-pair can we pick which key will be private or public?
No, in general we cannot. For most asymmetric cryptosystems the private and public keys are completely different kinds of objects (e.g. one may be a number and the other a point on an elliptic curve), and there's no way to use one in place of the other.
There is, however, one notable quasi-exception: the RSA cryptosystem has public and private keys that can be represented as the same kind of object (a modulus and an exponent) and used in the same mathematical operation (modular exponentiation) to encrypt and decrypt messages. So, in theory, one could generate a "symmetric" RSA keypair (as described in this question) and then choose which half to publish and which half to keep private.
In practice, though, even RSA key generation doesn't work that way, for a couple of reasons:
It's more efficient to choose the public exponent to be a fixed small number with a simple form in binary. (Common choices are 3 and 65537 = 216 + 1.) This is safe to do for the public key, but would obviously be disastrously insecure for the private key (since the exponent is the only secret part of the private key — the modulus is the same for both halves of the key pair).
There are ways to speed up RSA private key operations (i.e. decryption or signing) by using additional mathematical tricks such as the Chinese remainder theorem, but these require extra information about how the keys were generated (such as the secret prime factors of the modulus). Storing this extra information alongside the private key is no problem, and most commonly used RSA private key formats indeed do so. But publishing it would allow anyone to break the keypair and to decrypt and/or forge messages.
The result of all this is that, in practice, it's possible to compute an RSA public key from the private key, but not vice versa. As the linked answer (which I wrote) on security.SE says, it would theoretically be possibly to generate a secure private RSA key from which the public key cannot be computed, but this would require using both a nonstandard key generation algorithm and a nonstandard private key storage format. And even if the format issues were dealt with, not all RSA implementations would work with such private keys (due to the lack of the extra information mentioned above).
Do we encrypt or decrypt a Cipher Text with a given key? Is this technically correct to say so?
Sort of, yes. But what we usually do is generate a random key for a symmetric cipher such as AES, encrypt the plaintext with that, and then encrypt the AES key using the asymmetric encryption algorithm and the intended recipient's public key. And then we send both the AES ciphertext and the asymmetrically encrypted AES key to the recipient, who can first decrypt the AES key and then the ciphertext.
The main reason for using such hybrid encryption is that symmetric ciphers like AES are designed for encrypting lots of data fast, while asymmetric systems are not:
Asymmetric encryption tends to be relatively slow — often dozens or hundreds or thousands of times slower than encrypting the same amount of data with a fast symmetric cipher like AES. Asymmetric decryption is often even slower (whereas AES decryption is usually just as fast as encryption).
All secure asymmetric encryption schemes produce ciphertext that is longer than the plaintext, due to the need to inject some randomness into the encryption process to prevent guessing attacks. Some (such as ElGamal-like schemes) will double the length of the plaintext (after padding it up to the message block size). Others, such as RSA, require padding the plaintext up to dozens of times its original length. Obviously, this is undesirable when encrypting large mounts of data. Symmetric ciphers such as AES, meanwhile, generally only add a few dozen bytes of extra data to the ciphertext regardless of its length (depending on the mode of operation).
More generally, asymmetric encryption schemes are simply not designed to encrypt messages longer than a few dozen bytes or so (depending on the key size and other parameters of the scheme). An AES key will fit fine, but even a single chat message won't — and a whole web page or an image or a video stream definitely won't.
(Technically, the same is kind of true of symmetric block ciphers like AES, too: they also encrypt data in chunks of a few dozen bytes, and to encrypt long messages, they need to be used with a suitable mode of operation that applies the cipher to the data block by block. But there are plenty of fast and provably secure block cipher modes of operation that most crypto software and hardware supports out of the box, whereas for asymmetric encryption schemes there are basically none. And if you tried to design and implement one yourself, the odds are it'd be neither efficient nor secure.)
Is it correct to say that:
Having this key pair, I can encrypt a message with my private key and then publish it. That anyone can decrypt the message with my public key means I have encrypted it with my private key, which means it must have been me who made the message, as only I have my private key. Encrypting data with the sender's private key we call an open message format, because anyone with a copy of the corresponding public key can decrypt the message.
No, not really. The private key in an asymmetric encryption scheme is for decryption, while the public key is for encryption. You generally can't encrypt anything with the private key, just as you can't decrypt with the public key.
However, there are also asymmetric digital signature schemes that allow you to sign data with a private key and to verify the signature with a public key, proving that the data has in fact been signed with the private half of the same key pair.
Some digital signature schemes are indeed based on asymmetric encryption schemes. In fact, a digital signature can, in a certain general sense, be regarded as a zero-knowledge proof of the fact that the signer could decrypt a particular ciphertext (typically derived from a hash of the message being signed) with their private key using some asymmetric encryption scheme, thus proving that they had access to both the message (or at least its hash) and the private key.
In particular, the RSA cryptosystem can also be used as part of a signature scheme (such as RSA-PSS), using the same kind of public and private keys and the same mathematical operations (modular exponentiation) as for RSA encryption. This (and early popular explanations of public-key cryptography) is where the common but misleading description of (RSA) signing as "encryption with the private key" comes from. Technically it's not entirely wrong, at least if you ignore all the other parts of the scheme (hashing, padding, etc.) except for the modular exponentiation at the core of the RSA algorithm.
But if you're willing to simplify things that far, you might as well note that both RSA encryption and decryption (and signing and signature verification) are fundamentally just modular exponentiations, so you might just as well say that they're all the same thing, and that RSA signing is really just RSA decryption applied to a special ciphertext. Which would also be technically true, in a way, and probably no more (or less) misleading than describing it as "encryption with the private key".
