Authenticating somebody means verifying who they are. This always involves out-of-band information because otherwise you don't even know them.
If you like security games try to distinguish between two people without knowing anything about them. ;)
Even a PKI is a separate channel.
You can consider different authentication factors which are usually split into
- Something the person knows (Password-based key agreement)
- Something the person has (PKI, their private key)
- Something the person is (like a fingerprint)
but in any case you need to somehow be able to verify the information you get using some out-of-band information.
Maybe the 3rd factor "inheritence" deserves a litte more attention. It also includes things that the person does. So you could require that your communication partner behaves in a specific way. However if that behavior is not used to identify somebody you are actually doing fraud detection or intrusion detection and not really authentication.
I suppose you could consider the case where you only know somebody from in-band interactions and you just want to make sure that it's always the same person. In this case you should just do a key exchange on the first interaction.