Encrypting credentials and reusing them securely

Question

I'm currently building a website where the users would enter their credentials for another web service that I'm going to scrape to get their data.

I want to make sure that when I save their credentials in my database, I'm using the best encryption possible and the best architecture pattern to ensure the highest level of security.

The first idea that I had in mind was to encrypt the data using an RSA pub key (PBKDF2, PKCS1_OAEP, AES 256bit... ???) and then allowing my scrapping script to use the private key to decrypt the credentials and use them.

• Could you recommend the right/best encryption to do this?

• If my server is hacked, the hacker would have access to both the database and the private key, since it will be kept on my server that runs the scrapping script and hosts the DB. I know that it's a difficult equation to solve. But maybe there is an architecture pattern that helps with this.

I'm coding in python and I believe PyCrypto is the go-to library for encryption. (Sorry I have very little knowledge about cryptography so I might be confusing technologies)

EDIT:

I'm also looking to find out how to design my server(s) to secure all this. I know that if I use RSA, I need to put my private key somewhere where it won't be reachable from internet if my server is compromised, but still the private key needs to be reachable by my scrapping script that needs internet access.

So as you can see i'm a bit lost. I'm ready to use two independant servers but I don't have any idea about the best system design to use here.

Many thanks.

Answer 1

The property you really want here is called Isolation.
You want all the three processes (Database, Scraper, Webserver) to run fully isolated from each other such that a compromise of the DB or the webserver doesn't end in a compromise of the passwords. So before we dive into how you can achieve this separation, let's just quickly assume we already established it and are bothered with the communication between the three modules.

Let's just quickly forget about the communication between the three modules except for when the passwords are involved (they're the most interesting part here). So you hand a public key to the webserver. Whenever a user enters his credentials for storage, you encrypt them using this public key and send the result to the database. Then your scraper will have the corresponding private key and fetch the encrypted credentials from the database, decrypt them and then fetch the data and store that in the database or send it to the webserver or whatever really.
As for the choice of primitives here: Use RSA-2048 with OAEP to encrypt a random AES key and IV and encrypt the credentials using this AES-GCM. Alternatively you may use RSA-OAEP directly on the data but this limits you to less than 200 bytes for the credentials. Alternatively you can consider using the OS's crypto services, a TPM, a secured SGX enclave or a proper HSM for private key storage.

---

But what does this fancy transport mechanism help you, when your webserver is compromised and an attacker can just read the private key out?
This is where isolation comes into play. Here are a few things you could do:

• Clearly specify what queries the scraper and the webserver can make to the database and enforce the limitation from each individually to these queries using the database's credentials.
• Consider logically or physically separating the services, ie by putting them all into different [chroot][8] jails, into separate VMs or running them on different machines.
• Prefer separating the database server and the scraper script context-wise to ensure that a database server compromise doesn't lead to a credential compromise.
• Depending on how often you restart the script, consider encrypting the private key of the scraper with a password, which you'd have to supply on every restart. To convert the password into a strong key, consider using Argon2d.