Is there any advantage – other than potential memory or speed performance reasons – of picking a state size different from 512? If, what would the advantage(s) be?
Yes.
With 256-bit state, the main advantages are memory use and hardware implementation area.
With 1024-bit state, a hardware implementation can be faster, but there are also security advantages.
The paper (pdf) claims the following, given state size $n$, output length $l$ and $m = \min\{n, l\}$. I've included only the ones where a 512-bit hash with 1024-bit state has a higher security claim than a 512-bit hash with a 512-bit state:
- $r$-multicollision resistance: $\min\{2^{n/2}, 2^{(r-1)m/r}\}$.
- Key recovery, forgery and distinguishing resistance (as MAC, PRF or in HMAC): $\min\{2^{n/2}, 2^{m}\}$.
None of these are likely to matter in practice unless Skein is broken, which is why they recommended 512-bit state. However, it is potentially useful if collisions are eventually found after all, and as a security buffer (same paper):
Even if one day finding collisions
turns out to be somehow feasible—as for MD5—exploiting that weakness for creating either multi-
collisions or meaningful collisions should remain infeasible. This requires us to increase the internal
state size, which is the core idea for the failure friendly “wide-pipe” design [77]. Thus, if we want
a 256-bit hash function to be failure friendly, we need 512 bits of internal state, and if we want a
failure-friendly 512-bit hash function, we need 1024 bits of internal state.
Finally, the 1024-bit variant has been defined with eight more rounds to counteract slower diffusion (80 as opposed to 72 rounds). This has the effect of making some other attacks break a smaller percentage of total rounds: e.g. the attack mentioned on page 67 breaks 25/72 rounds with 512-bit state but only 26/80 rounds with 1024-bit state.
(You could define an 80-round 512-bit variant that shared the above advantage, but that would likely be slower than the 1024-bit variant even in software, at least on fast 64-bit CPUs.)