Multi User limited Time Encryption of big Datasets

Original Question: (Thanks cypherfox for answering that one)

I am in need to store data > 100 TB on a local network share (NAS/SAN) and grant individuals access to the encrypted data. For all users, besides me, the data can be read only, and sequential access to the data (only one user can access the data at one time) is fine.

The biggest problem is that I need to revoke individual users access from time to time (in addition to the recall of the network share access). I could encrypt the master key with user individual keys but this would not prevent technical experienced users from stealing the master key right? So I would need to re-encrypt the data every-time a user is recalled. Is there a way to do this more efficient than copying all data to a new encrypted image/disk? For 100 TB this would need more than 9 days.

Maybe someone can give me an advise which system/tools/techniques I should use and where the decryption should take place (client/server). Here the summarized key facts:

• ~ 100 TB Data
• max 5 user at the same time or 20 over the whole time span
• data stored an a local NAS/SAN connected to an 'offline' network
• the files have easily multiple GB, so some kind of transparent decryption to avoid copying the data would be necessary
• even a user could copy the data while he is a regular user, and even if he has no network share access anymore, we need to depreciate his decryption access
• This kind of access control would be better managed with capabilities. Take a look at Tahoe-LAFS – cypherfox Mar 20 '18 at 4:46
• Although revocation is tricky in all cases. You cannot prevent someone from keeping a ciphertext (or plaintext) and its decryption key. But you can restrict the distribution of the ciphertext and revoke this authority to access the ciphertext than the ability to decrypt ciphertexts. – cypherfox Mar 20 '18 at 4:51
• Makes sense. Looking a litte bit into Tahoe-LAFS it seems like the Data has to transfered via SFTP which is incredably slow. So downloading a single dataset of 50 GB would need more than one work shift. What do you think about giving people the ability to access the encrypted data (with their key), revoke their ability to access the network and remove them from the smb share (with encrypted transportation) as soon as their time is over. Furthermore I thinking of some kind of dedicated machine and offline network wherere logical datatheft would be quite hard to accomplish? – Subcentos Mar 20 '18 at 7:25
• To clarify. I linked to Tahoe-LAFS because it divides keys for 3 roles: Read, Write, and Replicate/Store. You could attach another authority requirement to the replicate authority to restrict the distribution of ciphertexts such as an authority that you can add caveats and revoke: Macaroons. – cypherfox Mar 20 '18 at 7:31

I'd start by dividing the read, write, and replicate roles into 3 keys. Tahoe-LAFS does this using "semiprivate" keys, each no larger than an EC curve point. I.e. Using ed25519, these keys would all be 32-bytes. The write key implies read, and the read key implies the replicate/verification key.

Once these keys have been revealed to someone, they cannot be revoked. To implement revocation, we need an additional authority that may be revoked, such as Macaroons. Macaroons support both delegation (sharing) and provisioning (with additional restrictions). These restrictions may reference clocks, or limit the functions the user may call (I.e. to read, write, append, truncate, delete, ..), or query out to external services to either check if the credential is still valid (revocation) or to check if the message is approved by the authority (configurable filters).

You cannot force a user to delete any ciphertexts, keys/credentials, nor plaintexts. You can revoke credentials to limit the distribution of ciphertexts. You don't need to rotate the encryption key so long that your adversary cannot also obtain the ciphertext (a revoked user cannot).

If your trusted server is allowed to read the user's data, then you don't need encryption at all (aside from session transports) as you only care about the authority to access. If the users can create new documents, then you'll likely prefer to lock the server (and you, the administrator) out of the user' personal data.

• Sounds prmising at the moment but I have to digg a little bit deeper into your ideas as this is completly new to me. – Subcentos Mar 20 '18 at 9:46
• More or less it is my data. No user needs the right to write, its not their data and all of them should be able to see the whole dataset. I trust the server as it is mine, but I need a least encyrtion on the server side in case the drives get stolen. But unfortunately this wont help against pysical attacks on the server ports (usb..). – Subcentos Mar 20 '18 at 9:55
• Fair threat model. So you can skip the tahoe-lafs part and just use macaroons if you need the sharing. Otherwise if you have say SSH/SFTP, each user can just have an account that you can destroy ? – cypherfox Mar 20 '18 at 10:18
• Thanks a lot for helping! Yes, I can make my own server so indivudal accounts for sharing wouldn't be a problem (I will make a local network as well but I can not protect the ports). SSH/SFTP is unfortunately way to slow, thats the reason I wanted to use SMB3 with Encryption. Would it be sufficient to encrypt the data, decrypt it only on the client side (transparent image or so with encrypted network SM§ transportaion) to prevent data theft from physical attacks on the server? – Subcentos Mar 20 '18 at 12:57
• How is ssh/sftp too slow? Is it because you need to work with many files? If so, then you can create an archive and fetch that, then extract it. I.e. ssh user@server tar cJf - foo bar baz | tar xf - to send the files/directories: foo, bar, and baz. – cypherfox Mar 20 '18 at 13:37