might have the terminology wrong when I say "GF(2) polynomial multiplication"
You are thinking of multiplication in the ring of binary polynomials, that is polynomials with coefficients in the Galois Field with 2 elements. That set is noted $GF(2)[x]$. It's addition reduces to XOR of the coefficients of equal weight. It's multiplication is called "carryless multiplication" in computing. It supports an analog to Euclidean division, and modular arithmetic.
It's my understanding that if you calculate $N=pq$ using (this) multiplication rather than ordinary multiplication, it is easy to factor $N$.
Yes. There are efficient algorithms for factoring binary polynomials into irreducible polynomials (which are analogs to primes in the ring of binary polynomials).
Does it mean that the security of RSA comes entirely from the addition carries that happen during ordinary multiplication?
True, RSA is not secure if we remove the carries (in the additions within it's multiplication by the schoolbook algorithm). I can't agree with "security .. comes entirely". Analogy: a chain is insecure if one removes it's last link, yet it's security does not come entirely from that link.