To understand the Even-Mansour security result, it helps to first understand the random oracle model. Roughly speaking, the random oracle model says that a hash function should behave like a randomly chosen function, and have no detectable patterns or structural properties that a random function wouldn't have. Of course, the hash function is publicly known, so anyone can compute it, but apart from that, it shouldn't have any other structural properties.
Even-Mansour uses something similar, but now for permutations. The requirement in the Even-Mansour proof is that $F$ is a random permutation oracle: it has to behave just like a randomly chosen permutation (random bijective function), and have no detectable structural properties that a random permutation wouldn't have. Of course, $F$ can be publicly known (so given $x$ anyone can compute $F(x)$ and $F^{-1}(x)$), but apart from that, it shouldn't have any other structural properties.
This is like an unkeyed version of the ideal cipher model. See What is the ideal cipher model?.
Once you understand the ideal cipher model, you can then evaluate whether some specific construction would qualify. It basically comes down to whether your other construction qualifies as a reasonable instantiation of a random permutation oracle (i.e., as a reasonable instantiation of an unkeyed version of an ideal cipher).
Fisher-Yates is not suitable except for constructing block cipher, as the running time of Fisher-Yates is super exponential: if you want a random permutation on $b$ bits, the running time of Fisher-Yates is at least $(2^b)!$, which grows asymptotically as something like $2^{(b-1.44) \cdot 2^b}$... i.e., huge.