I am writing an application, running on a server, where multiple users access data from a database which is AES encrypted with a master secret. The master secret itself is initially randomly generated, and then AES encrypted with a user-secret to yield a 'user-hash'. The master secret is never stored, but the user-hash is stored in a database. When a user enters his user-secret, the user-hash is decrypted to temporarily (on the fly) yield the master-secret which is used to decrypt the data. The data is then send to the user.
If a user is deleted, the user-hash is deleted. He may have saved decrypted data, but cannot decrypt further data.
The question is: How safe is the master-secret in this constellation? If the database is compromised, an attacker has access to the user-hash and the encrypted data. If the attacker was a user, he may also have decrypted data. Will he then be able to break the master-secret?