How to avoid a chicken and egg scenario with encrypting passwords?

Question

I am working on application that allows users to upload files containing company data and then share those files with a list of other users that have specific roles within the system.

I want to encrypt the uploaded files to protect the data at rest while allowing the files to be shared by people who do not know the encryption password. The uploaded files are owned by the company that they belong too not by the user who uploads them.

• When a user creates an account on the system a private-public key pair is generated for the user, the private key is stored in the database and is encrypted with the users clear text password, which is accessible during the login process. User passwords are salted, , stretched hashes so other than during login the app does not know the clear text of the users password.
• When a user log's into the system successfully the encrypted user's private key is pulled from the database, decrypted and the private decrypted key is stored in the http session. I am assuming that while the decrypted private key is in RAM it going to be very hard but not impossible for a hacker to get a private key out of the RAM of a running process.
• When a user uploads a file the following happens:
  • A secure random UUID is generated and used as the password to encrypt the uploaded file using AES 256
  • The generated UUID is then encrypted with the user's public key and stored in the database.
  • The system determines a list of every user that can access that file
    • For every user that can access the uploaded file the generated UUID is encrypted with that user's public key and the encrypted UUID is stored in the database.
• When a file is being downloaded here is what the system does.
  • It locates the encrypted password for the file being downloaded
  • Decrypt the encrypted password using the user's private key
  • decrypt the file using the file password
  • return the clear text of the file to the user

To me this seems to be more secure than just storing the files on the server unecrypted a hacker who steals a copy of the data would not be able to read the files, since they would have to figure out the clear text passwords of the users before they could get at the user's private key.

The problem is dealing with password rests, since the users private key is encrypted with the users password if the user forgets their password and does a password reset they would loose their private key and thus access to the files they uploaded. Since I need to have a password rest feature in the application I am into a chicken and egg scenarios because how do I safely store the private keys of the users without end result being less secure than doing nothing?

Is there any way to store private keys that are encrypted with a users clear text password and still have a way when resetting a password to preserve access to the users data?

I am not a security expert and would prefer to use a known solution / library for this problem but I can't seem to find any so I have to make my own solution.

Answer 1

Short answer: the problem you're trying to solve can't be fully satisfactorily solved.

With the assumption that an adversary control your system to the point of being able to read the RAM containing secret keys, you won't be able to define a secure system. The closest thing to a real practical solution is to bring in a trusted execution environment. HSMs (not the standard PKCS#11 breed, the programmable ones) can do that, and might allow you to be safe from a compromise of the rest of your system (with one important vulnerability that needs a fix: you should not "return the clear text of the file to the user", but instead encipher it for that user using a session key, all within the trusted execution environment). Smart Cards (again programmable, including Java Cards) could work in theory, but throughput considerations will most likely prevent doing the bulk of data encryption this way.

Notice that no matter what, the data can't be safe from the leakage of the credentials of any user that can access it, and this is likely to be a practical weakness offsetting vulnerabilities of the server. It might be possible to fix that partially, by limiting the amount of data a single user can obtain, or/and using a Smart Card on the user side.

I think the question would be most appropriate on security.stackexchange.com.

Answer 2

As fgrieu notes, the problem as specified is unsolvable: if the server alone should not be able to decrypt the files, then there must be something (in this case, the password) possessed by the user but not the server which is needed to decrypt them. If the user loses this extra information, there's no way the server can provide them access to the files — if it could, it could also provide itself with access.

However, let's step back and think about what a "password reset" really accomplishes. Basically, it's a mechanism that allows users to prove their identity and access their data using some proof of identity other than their usual password. For example, in the case of a password reset e-mail, the proof of identity is effectively access to a predesignated e-mail account, to which the password reset code is sent.

(There are various fundamental security concerns with that, as with most "password reset" schemes, but I'll ignore those for now and assume that having a password reset mechanism has been found necessary and desirable, and that the only issue is how to implement it. I do strongly recommend that the user should be able to opt out of any such mechanism in advance, if they wish to do so in order to minimize the attack surface for their account.)

So, instead of having the server send the user a single-use password reset code after the user has forgotten their password (which is impossible in this case), you could allow the user to request one in advance, to be saved in case they need it later. Indeed, you could even send one automatically (unless the user explicitly opts out) as part of the signup process, and resend a new one whenever the previous one has been used.

In effect, you'd be allowing your users save a back-up key to their account. I should note that this is not a new idea: for a recent example, the Firefox Sync service does something quite similar.

What should this back-up key contain, then? Well, obviously it needs to allow the server to access the user's internal key, just like a normal password would. It probably should not be the user's normal password, though, or anything related to it, just to avoid concerns about password reuse. Also, if you want the back-up key to be single-use, it should not simply be a copy of the internal key either.

One simple solution would be to just generate a random password (of sufficient length), encrypt the internal key with the password, save the encrypted key at the server and send the password to the user as their back-up key. Then the only difference between this back-up key and the user's normal password would be the actions the server takes when the user logs in with the back-up key (typically, request a new password, generate and send a new back-up key, and replace the old ones so that they can't be used again).

Answer 3

The answer to the question:

Is there any way to store private keys that are encrypted with a users clear text password and still have a way when resetting a password to preserve access to the users data

is actually yes. Simply store two copies of each private key - one encrypted with the user's password and one encrypted with the administrator's public key. Since "The uploaded files are owned by the company", there is no privacy issue with the fact the administrator can access all the uploaded files.

I am not a security expert and would prefer to use a known solution / library for this problem but I can't seem to find any so I have to make my own solution.

Have you tried PGP?

I agree with fgrieu that this question belongs on security.stackexchange.com.

Answer 4

Almost a decade later and the same challenges exist.

We came up with more or less exactly the same solution as the author. We have pub $+K_n$ and priv $-K_n$ keys pr user $n$. Private key $-K_n$ is stored encrypted $E_{K_S}(-K_n)$ in the database. Private key $-K_n$ is used to decrypt $D_{-K_N}(K_d)$ any shared key $K_d$ user $n$ are authorized to access.

The difference is that we have a system key $K_S$ which is used to encrypt users private keys $E_{K_S}(-K_n)$ when needed. The system key $K_S$ is provided at application startup (not exactly since it is decrypted in a config file at startup by a system provided password. In theory the same).

If a user want to access encrypted $E_{-K_n}(d)$ data $d$ he/she is authorized for they must reauthenticate with their user domain password. If authenticated the system will decrypt users private key with the system key $D_{K_S}(-K_n)$, then decrypt the data key $D_{-K_n}(K_d)$ and lastly decrypt the data $D_{K_d}(d)$. A long chain of events but it works.

There is two advantages by using domain passwords (corp ldap). First every user will follow any company security policy (password requirements, password changes, etc). Secondly users changing their domain password will not interfere with users stored private key. Users can renew their password without affecting the system. Decryption only require a reauthentication.

Some might question admins access to the system password and that hey can access any encrypted data. You are of course right. The solution is not design to keep admins away. Admins/developers can assess everything anyways.

The best would of course be if all users could handle their public and private keys in a safe manner on the outside of the application. Lets face it. That would probably be a complete failure with missing keys and keys saved in every digital corner. Still admins would be able to intercept the private key whenever entered by users. The system can be seen as more vulnerable by having private keys stored in the database and the system password in memory at runtime but giving keys to all our users would probably be a greater security threat. Also it's an intranet application behind more firewalls and crypto units than I can count for.

So sum up:

1. System password provided at application startup
2. Users have one password (corp ldap) and can change it whenever without affecting the application or the decryption process
3. One keypair per user stored in the database (RSA 2048)
4. Users private key encrypted with system password (AES/GCM 256)
5. Data key encrypted with users public key (RSA)
6. Data encrypted with a Data Key (AES/GCM 256). One key per data/document. Key can be shared with multiple users
7. Users are required to reauthenticate to start the decryption process