It is the IV as a whole that needs to be unpredictable... but knowing some bits would make guessing the IV easier.
The requirement of an unpredictable IV has to do with chosen plaintext attacks. If the attacker can predict the IV, they can choose the first block of plaintext such that they can verify any guesses they have for the content of previous messages. For this to work, they must be able to end up with exactly the right input to the cipher – even a one-bit difference will generate a completely different ciphertext block and not verify the guess. (It would verify a slightly different "guess", which might sometimes be important, so getting very close to predicting could still be a problem.)
By using some bits of the IV on the predictable IP address, you make it that much easier to guess the rest. If the other 96 bits were completely random, the attacker would have a $2^{-96}$ chance of correctly guessing it, which is uselessly small. If it was less random then the attacker could have a chance.
By reducing the random part of the IV you also make IV collisions more likely (between IVs of the same device). The situations where this would be helpful are few. For example if you have a good CSPRNG but equal/close seeds on the different devices. If that was the case you would be better off seeding the CSPRNG with the IP/MAC in addition. Or using deterministic derivation of IVs as CodesInChaos suggested.