The question is for the standard 256-bit Elliptic Curve secp256k1. It wants the key in the format described in sec1v2, section 2.3.3 Elliptic-Curve-Point-to-Octet-String Conversion, with point compression, that is per 2.1/2.2.1/2.3/2.4
To decode the DER-encoded public key SubjectPublicKeyInfo or X.509 certificate, one can use this online tool, which accepts hex and base64. Most likely it will show the $X$ and $Y$ coordinates that have been encoded as ASN.1 integer in the public key.
The desired output with start with byte 02
or 03
(with the same parity as the $Y$ coordinate), followed by the $X$ coordinate in big-endian notation over 32 bytes. Conversion to hexadecimal makes it a 66-character string.
When making this programmatically, the textbook solution is to use an ASN.1 library. When doing without, be careful that the ASN.1 encoding of integers has variable length, so that it may be needed to remove one 00
(≲50% of the cases) or add one 00
(<0.4% of the cases), or even more (<0.002%).
Note: After parsing $X$ and $Y$ from the DER-encoded public key SubjectPublicKeyInfo, it's a good idea to check that they form a valid secp256k1 public key, that is: for $p=2^{256}-2^{32}−977$ (the prime field order) it holds
- $0<X<p$
- $0<Y<p$
- $X^3-Y^2+7\bmod p=0$