It seems like there are different kind of private key formats, namely:
I have seen these names of keys thrown around in articles and answers I read while searching. What're the differences between these keys formats? Which one should I know I should use? And is the encrypted private key actually more secure than the plain version of private key?
There is no such thing as a PKCS#5 plain private key. PKCS#5 defines Password Based Encryption (PBE) and it is noted that this can be used to encrypt a private key.
PKCS#8 does define plain and encrypted private keys. By using a parameterized object identifier that indicates one of the PKCS#5 schemes you can encrypt the private key.
Which one you should use if any is up to you and your requirements.
Encrypted private keys can be used to perform password based decryption where the encryption is performed by the corresponding (unencrypted) public key. The other party in other words don't need to know the password, they just need to have and trust the public key. Arguably PGP would be a better defined / more compatible format for that though.
Another advantage is that the password can be used to encrypt the private key so that the private key can be stored on e.g. a backed up network connection. This would be secure as long as the password used is relatively secure (note that dictionary / brute force attacks are possible in this kind of scheme).
Of course there are certainly other and more secure ways to protect a private key. Plain private keys should normally only be used for testing, although storing them on separate media (a thumbdrive for instance) could give some protection.
