Yes, the initialization vector is transmitted in the clear as first part of the ESP payload data (see RFC 4303, section 2, and the RFCs that define the use of specific algorithm with ESP, e.g. RFC 3602, section 3 for AES-CBC, or RFC 4106, section 3 for AES-GCM).
The security of the encryption and integrity protection does not depend on the IV being kept secret. Its primary use is that of a nonce to ensure encrypting the same data with the same key does not result in the same ciphertext.
Some algorithms require that the IV is random/unpredictable (e.g. AES-CBC) but in many modern combined mode algorithms (e.g. AES-GCM or ChaCha20/Poly1305) it just must not repeat and can simply be a counter.
Under certain conditions it might not even be necessary to transmit the IV, for instance, because it can be derived from other information, like the sequence number in ESP. Therefore, there is currently an internet draft under discussion that proposes to omit the IV for certain algorithms (e.g. AES-GCM) and instead just use the (extended) sequence number, which is what's generally already used anyway, it's just transmitted twice in each message.
