Why a field: The idea behind Shamir's secret sharing is both to reconstruct the secret (functionality) and prove that any shared secret is possible (security) using polynomial interpolation.
While polynomial interpolation can be done over many algebraic structures, it will always work over a field. (Over a field, a non-zero polynomial has at most as many zeros as its degree. Over other related algebraic structures, this is usually not true.)
While Shamir's secret sharing is usually done over fields, it has been done over many other algebraic structures. This usually requires great care and is complicated. Unless you really have to, doing it over fields is much easier and preferable.
Why finite: It is not sufficient for security that any shared secret is possible, every shared secret must also be (almost) equally likely. Using finite fields allows us to choose randomness from a uniform distribution, which turns out to give us exactly what we want.
We could work over an infinite field such as the rational numbers, but in this case it would be very difficult to get every shared secret to be almost equally likely. This is related to not having a uniform distribution on infinite sets. Roughly speaking, one way to look at it is that the size of the value is related to the size of the coefficients and where we evaluate, so if we want to hide one of the coefficients, we need to "drown it out" by having the other coefficients be much larger.
Doing it over the integers (not a field!) can be done, but getting to security requires quite a lot of work. As a side effect (at least for the scheme I have looked at), the shares end up being much bigger. You do not want these costs unless you have a good reason. (Which you do, sometimes.)
We could try to work over an approximation of an infinite field such as the real or complex numbers, but in this case things become much more complicated, since we must also deal with inexact arithmetic. I have not seen anyone trying to do this, except by mistake.
Other areas of cryptography: Finite fields are used all over cryptography. Typically, this is related to non-zero elements in fields having multiplicative inverses, which we very often want. The operation also has many other nice, provable properties.
The finite part is usually required for practicality, and sometimes because of particular properties of finite fields.
AES: One example is in the AES sbox, where many desirable properties follow from the algebraic properties. You would not get the same algebraic properties from the integers modulo 256 (a ring), for example.
Multiplicative subgroup: Another example is the multiplicative subgroup of the finite field (the non-zero elements of a finite field form a cyclic group), which for a carefully chosen finite field turns out to be a suitable group for d.log.-based cryptography. (Discrete logarithms are defined in a similar way to ordinary logarithms, but it turns out that in some groups they seem to be very hard to compute without a quantum computer.)
In this case, we could also use certain rings, but it turns out that in practice prime finite fields are better for this sort of application. For instance, security does not depend on a secret group order, which allows us to do some things you cannot do if the group order is unknown. (RSA works over such rings, but has other properties and requirements.)
Elliptic curves: Yet another example are elliptic curves, which are extensively used in cryptography (even post-quantum crypto). While something very much like elliptic curves can be defined over other algebraic structures such as rings, the rich theory of elliptic curves requires working over fields.
The study of elliptic curves is an important part of number theory, but for cryptographic purposes curves defined over infinite fields are impractical or unsuitable for functional purposes and do not have the required security properties. (For example, an approximate discrete log can be computed by looking at the size of the coordinates, which would have broken security, had it not first comprehensively broken practicality.) Even though elliptic curves defined over infinite fields are not used functionally in cryptography, their study is essential for the analysis of elliptic curve cryptography.
Elliptic curves over certain finite rings have been considered in cryptographic contexts, but except for obscure cases do not offer anything of interest. (Elliptic curve factoring obviously excepted!)
Further examples: Lattice-based cryptography and code-based cryptography, which uses algebraic structures defined over finite fields. Multi-variate cryptography, which is based on systems of polynomial equations over finite fields.
Again, much of this can be done over certain finite rings, but there are many disadvantages and not much to be gained.