Your approach can only be applied to RSA and only if you are using keys generated according to PKCS#11 version >= 2.40. The PKCS#11 standard defines (section 2.1.3 "RSA private key objects" in the 3.0 base specification): "Effective with version 2.40, tokens MUST also store CKA_PUBLIC_EXPONENT."
Therefore, you're able to get the modulus and public exponent attributes (C_GetAttributeValue) and create a public key object from it using C_CreateObject. But as DannyNiu pointed out this certainly costs performance.
Depending on the token you might be able to use the private key object instead of the public key for operations such as C_Encrypt* or C_Verify* since the token will just use the public attributes.
For other asymmetric keys, eg. EC keys, there is no definition similar to the one for RSA in the PKCS#11 standard. Therefore, no public parts will be stored. It will depend on the token if the public key is calculated on the fly (confirming Conrado's comment). This certainly has an even higher performance impact. If the token is not doing it automatically, there is no way to trigger the calculation of a public key from a private key from PKCS#11. And of course you should never do it externally - why would you use a token (HSM) at all then if you use the private key in plain externally?
Finally, if you use standard applications your approach will likely fail since applications often add CKA_CLASS=CKO_PUBLIC_KEY to the search template when they look for the public key they want to use.