Many cryptographic algorithms are expressed as iterative algorithms. E.g., when encrypting a message with a block cipher in CBC mode, each message "block" is first XORed with the previous encrypted block, and the result of the XOR is then encrypted. The first block has no "previous block" hence we must supply a conventional alternate "zero-th block" which we call "initialization vector". Generally speaking, an IV is whatever piece of data is needed to begin running an algorithm, and is not secret (if it was secret, we would call it a "key", not an IV).
If you take a look at MD5, you see that it is an iterative algorithm which has a "running state" (four 32-bit words) and processes message data by 64-byte chunks, each yielding the next running state; the final state is the hash output. This has to begin with a conventional initial state, which is described in section 3.3 of the RFC.
Since an IV has some cost (e.g. it must be transmitted along a message), it is not there just for aesthetic reasons: algorithms which use an IV need it to fulfil some sort of security property, and this may imply some constraints on what value an IV may have. This really depends on the algorithm. For instance, with MD5, the IV is fixed and this is not an issue. For the case of CBC, see this answer to a previous question. Since uniform randomness is a difficult requirement (alea is a scarce resource, especially in embedded systems), it is considered a good thing if an encryption algorithm requires only a non-repeating IV (e.g. a simple counter); this is all that newer encryption modes such as EAX need.
An IV can be made public but nothing forces you to. It still tends to have a lifecycle distinct from that of a key, in that (for symmetric encryption) you need a new IV per message but not necessarily a new key. For instance, with TLS 1.2, a session key is created during the tunnel setup (the "handshake"), then data is encrypted as so many "records" (up to 16 kB of data per record) and each record has its own IV.