The initial hash values for SHA-512 are the 64-bit binary expansion of the fractional part of the square root of the 9th through 16th primes (23, 29, 31, ..., 53). That is:
$$I_0 = \left \lfloor \mathrm{frac} \left (\sqrt{23} \right ) · 2^{64} \right \rfloor$$
$$I_1 = \left \lfloor \mathrm{frac} \left (\sqrt{29} \right ) · 2^{64} \right \rfloor$$
$$\cdots$$
And obviously the reason they use primes is that their square root is always irrational, which makes for a nice, random-looking number. Taking the square root of four would be less interesting, as it would give 0. It also means the constants are relatively independent from each other, as none of them can be easily expressed in terms of any other, which might be a weakness - but that probably doesn't matter as much.
They are chosen this way because while they generally don't need to be anything in particular, as long as the chosen numbers don't have some special properties (i.e. setting them to zero might be a bad choice), the designers of SHA-512 wanted people to know that the numbers weren't chosen so as to stealthily introduce a backdoor in the algorithm (for instance, if a certain combination of initial values were chosen, it may have been possible for the NSA to break the hash function more easily) which would be very difficult to confirm or refute as it would require months, perhaps years of cryptanalysis.
Using publicly available numbers that anyone can recreate, here the fractional parts of some prime square roots, solves this problem - there is no way anyone can change these numbers (nobody can change the square root of 23, for instance), and it is considerably harder to design an algorithm to have a backdoor under set constants, than to design the algorithm first and work out those constants later on.
The first few digits of $\pi$ or $e$ are popular choices too, but only if you use the first digits. If you were using, say, the digits starting at position 94858991, I would get very suspicious.
See Nothing up my sleeve number, also this similar question.