in my mind there must be other points that exist that still yield $y^2 \bmod p=x^3 + ax + b \bmod p$ to be true but are never used

Actually, that's not true if the order of the curve is prime; examples of such curves are P256 and Sec256k1. In those curves, every single point can be expressed as $xG$ for some integer $x$.

Now, this is (usually) not true for curves with a cofactor > 1; in those curves, we generally work in a prime-sized subfield; there will be points that are 'missed'. To determine whether a point $H$ we've been handed is such a point, one way (which works with the curves we use in cryptography) would be to compute $qH$ (where $q$ is the size of the prime subgroup) - if that is not the neutral element, then $H$ cannot be generated by the generator.

Obviously, this is not a cheap check; what we usually do when working with a cofactor > 1 curve is to arrange things so that things don't break if we're handed a non-subgroup point.

in my mind there must be other points that exist that still yield $y^2 \bmod p=x^3 + ax + b \bmod p$ to be true but are never used

Actually, that's not true if the order of the curve is prime; examples of such curves are P256 and Sec256k1. In those curves, every single point can be expressed as $xG$ for some integer $x$.

Now, this is (usually) not true for curves with a cofactor > 1; in those curves, we generally work in a prime-sized subgroup; there will be points that are 'missed'. To determine whether a point $H$ we've been handed is such a point, one way (which works with the curves we use in cryptography) would be to compute $qH$ (where $q$ is the size of the prime subgroup) - if that is not the neutral element, then $H$ cannot be generated by the generator.

Obviously, this is not a cheap check; what we usually do when working with a cofactor > 1 curve is to arrange things so that things don't break if we're handed a non-subgroup point.

in my mind there must be other points that exist that still yield $y^2 \bmod p=x^3 + ax + b \bmod p$ to be true but are never used

Actually, that's not true if the order of the curve is prime; examples of such curves are P256 and Sec256k1. In those curves, every single point can be expressed as $xG$ for some integer $x$.

Now, this is (usually) not true for points with a cofactor > 1; in those curves, we generally work in a prime-sized subfield; there will be points that are 'missed'. To determine whether a point $H$ we've been handed is such a point, one way (which works with the curves we use in cryptography) would be to compute $qH$ (where $q$ is the size of the prime subgroup) - if that is not the neutral element, then $H$ cannot be generated by the generator.

Obviously, this is not a cheap check; what we usually do when working with a cofactor > 1 curve is to arrange things so that things don't break if we're handed a non-subgroup point.

in my mind there must be other points that exist that still yield $y^2 \bmod p=x^3 + ax + b \bmod p$ to be true but are never used

Actually, that's not true if the order of the curve is prime; examples of such curves are P256 and Sec256k1. In those curves, every single point can be expressed as $xG$ for some integer $x$.

Now, this is (usually) not true for curves with a cofactor > 1; in those curves, we generally work in a prime-sized subfield; there will be points that are 'missed'. To determine whether a point $H$ we've been handed is such a point, one way (which works with the curves we use in cryptography) would be to compute $qH$ (where $q$ is the size of the prime subgroup) - if that is not the neutral element, then $H$ cannot be generated by the generator.

Obviously, this is not a cheap check; what we usually do when working with a cofactor > 1 curve is to arrange things so that things don't break if we're handed a non-subgroup point.