Let's say I want a way of generating a deterministic public ID for a specific symmetric cryptographic key (AES, for example). Would it be secure to use SHA-3 (or SHAKE) to hash the key, and then distribute the hash as the public ID for the secret key?
Is there an advantage or disadvantage to using a larger or shorter hash length relative to the key length? For example, if the key is 256 bits (32 bytes), using a 256 vs 512 bit SHA3 hash?
Would it be secure to use SHA-3 (or SHAKE) to hash the key, and then distribute the hash as the public ID for the secret key?
Sure; SHA-3 and SHAKE are effective uninvertable. Now, I suppose someone could try various AES keys and see which hashes to the public identity; on the other hand, they could try various AES keys and see which one decrypts the ciphertext properly. We believe that the second is infeasible, and so the first (which requires no less effort) is also infeasible.
Is there an advantage or disadvantage to using a larger or shorter hash length relative to the key length?
The only disadvantage to using a shorting hash length is that it increases the probability that two different keys that you actually use will end up with the same public identifier (hash).
Essentially, the question is whether an AES instance $\mathrm{AES}(k, \cdot)$ would stay pseudorandom, if the hash $H(k)$ of the secret key $k$ is leaked. Certainly, this would reduce the entropy of the key, especially when the hash length is large. But, even when $H(k)$ is explicitly given, recovering $k$ would be still hard, and as poncho has noted, perhaps essentially not that different from bruteforcing the AES key.
This kind of security under leakage has been studied before. For example, the following paper by Bellare et al. studies what kind of things can be built, if we have a PRF which remains secure if some bits of the secret key $k$ are leaked, for example. That is, in this case, $H(k)$ is not even a hash function, but much simpler one, like truncation of $k$ up to some bits.
https://eprint.iacr.org/2016/142
While I'm not sure whether the security of AES under this kind of leakage has been concretely studied before, I think it is not that unreasonable to imagine that AES would remain secure if some $l$ bits of the secret key are revealed in this way (since there are still $128-l$ bits remaining hidden). So, perhaps, you might even simplify things and just use truncated key as the 'public ID', especially when your block cipher has large key length to spare. Of course, this would depend on your purpose, and also this would be somewhat less secure than using hash as the public ID, but at least this has the benefit of being more efficient.
