Indeed, the shared secret in Curve25519 is 256-bit. It can take about 2252 32-byte values. Notice that 256-bit values for 128-bit security make sense in many contexts. For example for 128-bit security against collision, a hash needs to be about 256-bit.
To turn that shared secret (or more generally the outcome of Diffie-Hellman, ECDH or not) into 128-bit keys for symmetric cryptography, the academic thing to do is to feed a Key Derivation Function this secret and a constant characteristic of the intended usage of the key, to obtain a 128-bit output. The KDF needs not be purposely slow. This key derivation serves two goals:
- We get as many keys as needed from a single shared secret. Often, in the context a sessions, we need at least one encryption keys in each direction as a simple way to block mirror attacks; perhaps two if we use separate encryption and integrity keys. We might also need an extra value to be signed, in order to protect the session against MitM.
- The output of a KDF appears uniformly random, when the shared secret of (EC)DH is often not, thus merely extracting bits from the secret is not ideal. If you extract the wrong ones, security drops (I think, by about 1 bit in the worst case for Curve25519 if the other bits are dropped, but don't bet the house on that).
The poor man's KDF is a truncated hash. You won't be bitten by using the first 16 bytes of $\operatorname{SHA-256}(\text{DerivatonConstant}\mathbin\|\text{SharedSecret})$ where $\text{DerivatonConstant}$ is some 8 bytes characteristic of the intended key usage. A more academic one is truncated HMAC-SHA-256 with key $\text{SharedSecret}$, data $\text{DerivatonConstant}$. NIST has a standard for KDFs.