I'm reading a handout about worst-case to average-case reduction by Peikert and Banerjee: https://github.com/cpeikert/LatticesInCryptography/blob/main/lec11.pdf
In the reduction from SIVP to SIS, we have a hypothetical oracle $\mathcal{O}$ for $\mathsf{SIS}$, by which we manage to solve an instance of SIVP.
I don't understand why we need to prove that the input matrix $\mathbf{A}$ must be close enough (by smoothing) to a uniformly random $\mathsf{SIS}$ instance, i.e., Lemma 3.5:
What's the problem if the matrix $\mathbf{A}$ is not uniform? Will the oracle reject such a query, and how would it find it? I understand that if we are interacting with an adversary, it may behave arbitrarily if the input we give doesn't satisfy the adversary's requirement, then we may not get what we want from this failure query. But if we are interacting with an oracle, does this mean that the oracle itself may call to some adversary? Why can we not just assume there is an oracle that can solve the SIS problem when given arbitrary matrix $\mathbf{A}$?