For this question, the following caveats and assumptions hold:
- There exists a 2048-bit RSA key pair used exclusively for signing/verification
- The private key is kept completely private
- There exists a group of users who have the public key
- There is a group shared password
- I (mostly) understand the functional difference between encryption/decryption and signing/verification for RSA, but not the specific math differences
- I am aware of hashing
Given the Following Operation
"ciphertext" = RSA_sign( public_key, random salt + password )
What are the security implications if using PKCS#1 old-style v1.5 padding?
Obviously, this is not the accepted way of doing things -- one does not sign with a public key. Basically, the end result is a deterministic "encryption" of the password which can ultimately be compared among the members of the group. That is, because there is no randomization of the padding, each member can perform the same procedure and arrive at a comparable result. So member A and member B (neither of which have the private key) can perform this operation (as long as they have the salt) and determine that the password is the same. (Hashing, of course, would be a much better solution.)
The above scenario is susceptible to known plaintext attacks. eg. Similar to a hash, a malicious user can attempt to brute-force the password by simply trying combinations until they get a comparable result (the salt is, for all intents and purposes, public knowledge).
Is the private key at risk (I don't believe it is)?
Is this weaker than a hash such as SHA1?
UPDATE: After thinking about the relative security of this "RSA-hashing" mechanism to SHA1 for collisions, I realize that there would be an increased chance if msg.len > RSA.n.
(Please don't tell me to just use hashing. This is not the point of the question. I have seen questions asking if it's okay to "encrypt" using the private key which is a clear security problem, but I've never seen an answer to the question of signing with the public key.)