I need a method to authenticate a process with another in order to establish interprocess communication between them, to prevent malicious processes from trying to hook onto the system. Currently I just send a password in plaintext between the two processes and the host process just performs a string comparison with its copy of the password, and will disconnect the connecting process if they don't match. It was fine when I was limited to connecting each process on localhost, but obviously this is not secure if someone just happens to have access to a network between each process and could sniff the packets. Do note that I only care about encryption for the password exchange and I don't really need it to secure the communication.
So basically I need a encryption algorithm that doesn't need to protect its contents well but does need to protect its key, and needs to be as fast as possible with minimal RAM usage and I/O overhead. Yes, data exchange is more CPU-bound than I/O bound since the two processes will be doing loads of calculations and they most likely will be connecting through localhost. Now, I came across RC4, which has a throughput of over twice that of AES-128, but it is evident that it has some weaknesses and I have a lot of questions about how to implement it in a secure way.
Well first question - it's in the title. Is RC4 completely broken and should be avoided? I hear a lot of "it is secure if properly used" and then a lot of "use of RC4 is discouraged."
I was planning to keep the plaintext password on both processes and have them SHA-224 hashed, and concatenate the hash with a 32-bit IV as a 256-bit key for RC4. Are there any security risks with this approach? Perhaps I could then hash the concatenated IV and hashed password with SHA-256? I'm well aware that WEP was cracked because it used a 24-bit IV, but a 32-bit IV has over four billion more unique values. Should I avoid incrementing the IV after each packet like WEP?
And when it comes to RC4-drop[n], what is a reasonable value for n? 256, 512, 768 bytes? How would you implement drop n on a library whose only public operation is encrypting a byte array - pass a byte array of length n after the engine is initialized with the key? Would that mean you would have to basically process n empty bytes after every time you change the IV (which happens after every packet)?
Sources: http://www.rsa.com/rsalabs/node.asp?id=2009
EDIT: after reading all these excellent responses, I realized how ignorant I sounded about the whole subject. I had no experience with cryptography before this day, and I only was able to read about RC4's risks from the limited Google results that I could barely follow. thank you to everyone who has been patient with me!
Correct me if I'm wrong, but I believe a better solution is to either use AES (which is not slower because re-keying RC4 after each packet is expensive) or an universal message authentication code. I can easily work with AES, but my understanding of MAC is a bit shaky. An IV is only secure if the update sequence is randomized and unpredictable, meaning I would have to send a 128 bit IV for every message that is sent, which can add a 400% overhead to the short, 4 byte packets that are commonly sent by my application. HMAC can add more than twice that amount if SHA-256 is used. I am going to have to investigate this matter further.