First of all, as already mentioned in the comments, there is absolutely no mathematics or cryptography involved.
Independent of the model used, i.e., public key cryptography, identity based cryptography, or certificateless public key cryptography (as i summed up in my answer here), the identity verification must be established by other means.
In the wild (currently) only public key cryptography is widely used and PKIX (PKI based on X.509 certificates) is the predominant "standard". Here, the user who generates the key pair sends a certificate signing request (including the public key, identity attributes - which should be included - and a proof that the user holds the corresponding private key - which is established by signing the request) is sent to the certification authority. As already noted in the comments, the "proof of possession" of the respective private key is required to be sure that the user does not come up with the public key of some other user (for which he does not possess the private key).
Then, how the identity verification is done before issuing the certificate mainly depends on the purpose of the certificate, i.e., the corresponding private key.
For instance, many countries, as widely encountered across Europe, have deployed national electronic identity (eID) solutions, where the respective private key can be used to sign documents, and such signatures are equivalent to handwritten signatures. There, you typically have an explicit verification of identity, e.g., by showing up with your smart card (which stores the private key) at a certified registration authority or by offering an online service for activation and the activation code is then send by traditional mail (which you can only receive by showing an official credential such as a passport).
However, this practice is quite different for SSL certificates (which makes up the biggest part) or software signing certificates.
Hopefully, this answers your question and I did not miss anything that should be said.