You're right, using RSA (or other public-key schemes) to sign and/or encrypt each message between the client and the server would be impractically slow, not to mention wasteful of bandwidth.
The standard solution to this problem is to use public-key crypto only to set up a shared secret key between the client and the server, and thereafter use a symmetric authenticated encryption for all communication between the client and the server.
Some methods of setting up the shared secret key also authenticate the client as part of the process. However, even if they do not, once you've set up an encrypted secure channel between the client and the server, it's easy enough for the client to prove its credentials over the encrypted channel without any eavesdroppers being able to observe the process.
This is basically what SSL / TLS does whenever you connect to a website over HTTPS, and indeed you could simply use a TLS library in your client and your server to establish a secure connection between them. Note that plain TLS is a stream encryption protocol usually used over TCP/IP; if your game's client-server protocol is packet-based and runs on top of UDP, you may want to pick a library that also supports DTLS. Of course, this would mean setting up TLS certificates for your servers, probably signed by your own private CA that your client code is configured to trust; all that is something of a chore, but still a lot easier and safer than writing your own crypto from scratch.
If you do insist on rolling your own crypto, or just want to get an idea of what it would involve, here's a quick sketch of something that should more or less meet your needs:
Each of your servers has an RSA public/private key pair. Each server also has a signature of its public RSA key, signed by your personal master RSA signing key. Each game client has the public half of your master key, allowing it to verify the authenticity of the server keys.
(The servers should not store the master key itself, since you don't want the master key to be leaked if a server is hacked. Also, the signed server keys should include a timestamp, and the client should not accept any signature whose timestamp is too old, so that any compromised server keys will eventually expire.)
When the client connects to the server, the server replies with their signed public RSA key. The client verifies the signature (and the timestamp), and if it's valid, generates a random AES key, encrypts it with the server's public RSA key, and sends it to the server.
The server decrypts the AES key sent by the client; after this, all further messages exchanged by the client and the server are encrypted and authenticated using AES-SIV. (All messages should also include a sequential message number and an indicator of which way they were sent, to prevent replay attacks.)
Note that the basic scheme above does not authenticate the client to the server in any way. If this is required, one simple solution would be to give each client its own RSA signing key, and having the servers store the corresponding public keys. After establishing a secure connection, the server can then send a randomly chosen challenge string to the client, and ask for the client to verify its identity (i.e. knowledge of the correct client's private RSA key) by sending back an RSA-signed response that includes the challenge string.
Other possible client authentication methods include e.g. SRP, or even just having the client send a password to the server over the encrypted channel. Note that the latter method should preferably be avoided (despite its ubiquity on the web) for the simple reason that it allows a compromised server to record client passwords. Schemes like SRP or even the basic RSA challenge-response scheme described above, where the server never actually sees the client's private key or password, are more secure in this respect.
