Well, lets go through the issues:
It seems to be possible to retrieve the (public) key used for creating an ECDSA signature just from the signature alone
Nope, not quite. You also need the message being signed. And, with that, it doesn't give you the unique public key; it does allow you to narrow it down to two possibilities (assuming you're using a curve with a cofactor of 1; almost always the case nowadays), however from a single signature, you don't know which of the two possiblities it is.
In some circumstances, the private key can be recovered as well
I should certainly hope not. If you can recover the private key with a signature and the message being signed (and you can throw in the public key if you want), then the signature scheme is "broken"; I rather doubt that Mr. Brown claims that ECDSA is broken.
this can be an advantage (shorter messages, as the public key or a key id doesn't have to be attached)
Well, it's not that much of an advantage; not only does the verifier need to know what the public key is, he also needs to know that it's the correct public key; that is, it is the public key whose private key is held by the person you think is signing the message. There are a number of ways a verifier can ascertain that; however he must do so; just passing a public key somehow (either explicitly or implicitly) doesn't cut it.
this can be a disadvantage (if confidentiality is required, the signature has to be encrypted)
Not so much; if confidentiality (rather, anonymity) is required (specifically, you don't want an evesdropper to be able to verify who signed the message), then you need to hide the signature anyways. If you don't, then the evesdropper does get a copy of Alice's public key, and has a guess for the message can simple check if the message is from Alice by running the signature verification method using his copy of Alice's public key. This doesn't require any special properties of the signature method.
Which other cryptographic signature schemes have that property?
I'll go through the ones you listed:
RSA; if the public exponent isn't large, and you use a deterministic padding method, then you can recover the RSA public key from two signatures (of two different messages)
Ed25519; I don't believe that you can recover the public key. The verification process involves a hash of data including the public key; there's no way to quickly check which public key would make the hash come out correctly.
DSA: I don't believe that you can recover the public key. The distinction between DSA and ECDSA is that the DSA verification process involves an outer "mod q" operation; that operation discards information that would be needed to recover the public key.