My only idea is that B authenticates himself to A, because if A later decrypts it, A will see whether B was able to decrypt it. But why would you need to increment the nonce?
Correct, that's the idea.
If B didn't need to increment the nonce and just encrypted the same value, the message sent back would be the same that A sent, so an attacker would be able respond correctly without knowing how to decrypt it.
There are other ways to accomplish the same in other protocols that do not use increments. For example, B could encrypt it with another shared key.
So are these two use-cases for the same purpose? Just to authenticate Bob to Alice? What is the difference between them? When would you apply which of these two and why?
In terms of authentication there is no real difference. They both prove to Alice that Bob has the correct key.
The one with encryption and increment is slightly less efficient, due to requiring both parties to encrypt and decrypt the values. However, it has the advantage of not allowing an attacker to directly ask for encryptions of chosen plaintexts, with which they could attack other uses of the same key. This is not necessarily a problem if the key is only used for authentication, but in the case of Needham–Schroeder the objective is to establish a session key.