Each device has a unique certificate with a signature (ECDSA).
Every certificate contains the public key of the certificate owner, and a signature of the certificate's content (including public key) by the certification authority that issued the certificate. "ECDSA" is somewhat unclear: we do not know if that applies to the certificate's public key (which would be typical) or/and to the certificate's signature (RSA is often used there for performance and historical reasons, but ECDSA can be used).
I'm unsure how the server selects which certificate to present for each device
If the server presents a certificate to the IoT device, that must be a certificate for a public key which matching private key is held by the server, thus there's no choice to be made upon certificates of different IoT devices, and typically for similar IoT devices and a single-use server there is no choice to be made by the server among multiple certificates that it may hold.
What are some typical methods for authenticating multiple IoT devices with unique certificates when connecting to the server?
During (at least the first) establishment of a TLS or HTTPS connection, the IoT device gives it's certificate to the server. The server checks the validity of the certificate the normal way (just like your browser does with the certificate of crypto.stackexchange.com when it connects here), and (as part of TLS) the IoT device authenticates to the server using it's private key (the server knows and trusts the matching public key which it got from the certificate). The IoT device's certificate contains information on the IoT device, which may include a serial number or/and the MAC address.
Perhaps the server also authenticates to the IoT device. If so, it works as above with roles reversed. That would be typical for a manufacturer's server. For a server that one setups and if the manufacturer's server is not involved, there is the issue of how the IoT device trusts that the server is authorized. Solutions vary for this.
