Curve25519 was chosen to have the Montgomery shape $y^2 = x^3 + A x^2 + x$ to support the fast single-coordinate Montgomery ladder for Diffie–Hellman: given $x(P)$ and $a$, it is cheap to compute $x([a]P)$, so there is no need to pass the $y$ coordinate and implementors are not tempted to use secret-dependent conditionals to compute scalar multiplication with the naive double-and-add algorithm—that is, the shape of Curve25519 was chosen to encourage implementors of the X25519 DH function to avoid timing side channels without sacrificing performance.
Every Montgomery curve has a point of order 2 (if the equation holds for $P = (x, y)$ then it holds for $(x, -y) = -P$ so $P + (-P) = \mathcal O$), so the cofactor is always greater than 1—and when the coordinate field is $\mathbb Z/p\mathbb Z$ for $p \equiv 1 \pmod 4$ as is the case for $2^{255} - 19$, the orders of the curve and its twist are 4 and 8 or vice versa.
Curve25519 was chosen to minimize the cofactors, and when taking all the other criteria into account, the best choice had cofactor 8 for the curve and cofactor 4 for the twist. RFC 7748 gives a deterministic algorithm to search according to these criteria in Appendix A. For comparison, the Curve448 coordinate field is modulo $p = 2^{448} - 2^{224} - 1$ with $p \equiv 3 \pmod 4$, so both the curve and its twist have cofactor 4.