The secret $s$ in ECDSA is a value in the range 1 and the order of the group (exclusive). Some parameters are chosen in such a way that you can simply generate any value within the amount of bits as the chance that you're outside of the range or choose 0 is very small indeed. The public key is a point on the curve, calculated by performing $w = s \cdot G$ - using (modular) point multiplication as $G$ is the base point of the curve.
Generally the value $s$ is just encoded as an unsigned statically sized, big endian integer of the same size as it takes to encode the order of the group in bytes. The public point $w$ however can be encoded in different ways. You've shown a simple concatenation of two unsigned statically sized big endian x,y coordinates. That means of course that it is two times the size of the secret $s$. To get the values of x and y you first have to split the public point encoding exactly in half and then decode the separated values of x and y.
Usually this "flat" encoding (i.e. without structure) is preceded by a byte with value
04 to indicate that it is an uncompressed point. It is however possible to create a compressed point, which is just a single bit larger than the x value. Unfortunately that bit is not available in the encoding of x, so generally the point x is prefixed by a byte with either value
03 (depending on the value of x and y). Then the value of y can be retrieved by uncompressing the point. So yes, you can "shrink" the public key.
Bitcoin is even sneakier and allows you to recreate the public key out of the signature. That's probably worth a separate question though.
Note that you must first trust the public key before you can use it to verify any signature. If you don't know who's public key you're using, then you do not know who's signature you're verifying either.