Well things like this, my coworker suggest exchange the private key between server and client and using public key to decrypt message privately, his reason is, this is just the same as normal usage, yeah for just encrypt and decrypt message they work the same. But I wonder is there any weakness in it, make it easier to crack?
No, this doesn't generally work:
- encryption / decryption could use another padding scheme if it switches depending on what kind of key is used;
- if the private key is leaked / made public then the public key can usually be guessed or calculated - this works at least for RSA and IES encryption using DH keys;
- the crypto library may not be able to switch around the public key and private key, for instance for RSA may require the CRT parameters and ECC keys are different altogether.
A wild guess: in 60% it would not be feasible, 35 percent it would be completely insecure, 4 percent just less secure and in 1 percent of the time hey, you could be lucky and have something that approaches secure, all depending on the protocol, algorithm and implementation.