For example, someone suggested improving UNIX authentication by defining 3 different passwords for a user that are stored in the shadow file. All three passwords are stored with the same salt value. When a user wants to authenticate to some system, he must provide in the i attempt to login, the i mod 3 password. For example, in the first attempt he will provide the first password, in the second and fifth attempt he will provide the second password. Is such alteration to UNIX system increases security or not?
I cannot see how it increases security. As long as you don't limit the amount of attempts the attack surface only increases.
what happens when all passwords are stored with different salt value?
You cannot see if one of the three passwords is identical to one of the other three. The reason to choose an identical salt is presumably to make sure that each password is different from each other.
Note that the passwords could still be very similar. The only way to check that is to compare the three passwords and for that you need all passwords at the same time.
It would also decrease attack speed by a factor of three (as an attacker cannot check a candidate password against all three hashes). This is of course assuming that the attacker obtained the hashes in the first place or that the password validation is sufficiently fast.