No, knowledge of the IKE preshared key will not allow anyone to listen into the IPsec traffic (either with PFS or without).
IKE always performs a Diffie-Hellman exchange to generate the initial keying material; knowledge of the preshared keys does not tell the attacker either the Diffie-Hellman private values, or the shared secret. The encryption keys are derived from the shared secret; hence they are secure as long as the DH exchange is.
The preshared key is present to act as a 'proof-of-identity'; what knowledge of the preshared key would do is allow you to act as an active man-in-the-middle (because to either side, the attacker is indistinguishable from the other side). The attacker would perform an IKE negotiation with both sides, generate traffic keys, and then be able to decrypt traffic from one side, read it, and then reencrypt it to send to the other side.