At what points do each of the four keys from the two key pairs get used?
Let Alice have a key pair (PubKeyA, PrivKeyA), where the first is public, the second private, and similarly for Bob (PubKeyB, PrivKeyB). Alice and Bob know each other's public keys in a reliable way (from a key server, or because they received them in person etc.) If we use RSA keys, e.g., then they could use just one pair, so I'll assume this first.
When Alice sends Bob a signed message (signing is optional in PGP, but good practice), the following happens, simplified:
Note that we need a secure mode of operation to AES to encrypt M using K, a secure way to encrypt the key K using PubKeyB (using a secure padding scheme, like OAEP), a secure hash function and way to sign (also using a padding scheme) and a good file format to unambiguously pack all the data in. Read up on the standards involving them...
Bob does the following (simplified) when he receives the message:
Now Bob has M too, and is confident it is the actual message Alice wanted to send, and was not tempered with in transit.
Like I said, the devil is in the details, but this is the essence of it.
Now, in practice, PGP creates 2 pairs per user, an encryption pair and a signing pair. Also the programs sends key-identifiers with the message to indicate which ones are used (in general, a user could have more public keys etc.)
Here's a mnemonic and metaphor for you.
Someone's public key is like their phone number. You have to have their phone number to call them and you use their phone number to call them. Other people use your number to call you.
From a crypto aspect, you encrypt to a public key and then decrypt with the private key.