6
$\begingroup$

I'm interested in two following processes:

1) Perform deep learning on homomorphic encrypted data

2) Perform deep learning predictions with a homomorphic encrypted model on unencrypted data. By this I mean encrypting weights of a deep learning model, sending them to the owner of the data, and perform an encrypted prediction. The owner of the data will return back to me the encrypted prediction. See for instance this blog post for an example with logistic regression.

I'm wondering if it's possible to do deep learning (with many hidden layers) with PHE (partially homomorphic encryption, e.g. Paillier) or if I need a FHE (fully homomorphic encryption).

References are welcome!

$\endgroup$
  • 1
    $\begingroup$ You might be able to do very simple things (e.g. linear regression) from Paillier and the like, but for deep learning, I can think of only two alternatives: SHE/FHE, or MPC (with interactions). $\endgroup$ – Geoffroy Couteau Feb 13 '18 at 11:33
7
$\begingroup$

There are very few (somewhat practical) results about homomorphic deep learning currently. As a good starting point, you might want to have a look at this recent paper from my former colleagues, and references therein. It focuses on optimizing lattice-based somewhat homomorphic encryption schemes, for evaluating deep neural networks.

$\endgroup$
3
$\begingroup$

In my opinion, the question's

1) Perform deep learning on homomorphic encrypted data

should not be considered without feedback from the holder of the decryption key. I have the intuition that it is impossible, and to my knowledge it was never even attempted. Learning is typically performed on clear data, and "learning on (..) encrypted data" seems to require at least some use of the decryption key.

That's because ciphertext yields no exploitable information for someone/something neither holding the key nor breaking the cipher. Without some access to the decryption key, any data processing on the ciphertext is nonsensical (except for cryptanalysis), even when the encryption is homomorphic. Homomorphic encryption will let one compute some function from the encrypted data, but the result stays encrypted, and no conclusion can be reached without the decryption key. I thus do not see how, without the key, decisions characteristics of deep learning activity (to connect a neuron or not, or other long-term change) could be taken meaningfully (better than randomly); or how positive learning feedback (where earlier learning enables new progress) could occur.

The best I see possible without the decryption key would be to compute encrypted values from encrypted training data, then use these encrypted values as parameters of a predefined model. That way, the encrypted data used for training would meaningfully influence the model. But is that learning?


Rebutal for comment, linking to Ehsan Hesamifard, Hassan Takabi, Mehdi Ghasemi's CryptoDL: Deep Neural Networks over Encrypted Data as argument that 1) can be done. Quoting that paper:

We assume that the training phase is done on the plaintext data and a model has already been built and trained

This (and other) literature builds deep learning networks that can process data encrypted with homomorphic encryption without holding the key; but I fail to see how something could meaningfully learn from well-encrypted data without using the decryption key, which is how I read the question's 1).


There is no such problem with 2), where deep learning was performed on clear data, and the model encoded for homomorphic encryption. Partially homomorphic encryption will not be very useful, for it won't allow the non-linearity present in any useful model. Fully Homomorphic Encryption seems a must.

$\endgroup$
  • $\begingroup$ 1) Should be possible, for instance: arxiv.org/abs/1711.05189 $\endgroup$ – Rexcirus Feb 13 '18 at 11:05
  • $\begingroup$ Even though of course I will get encrypted predictions out of deep learning. $\endgroup$ – Rexcirus Feb 13 '18 at 11:06
  • 1
    $\begingroup$ When someone talks about "performing homomorphic computation over encrypted data", it usually implies that the result will be an encrypted result from which nothing can be learned. The typical scenario, which I assumed was the one considered by OP, is the following: a client encrypts data, sends it to a server, who perform computation (e.g. evaluate a deep neural network), and send the result back, which the client decrypt. That's the standard setting of delegation of computation. $\endgroup$ – Geoffroy Couteau Feb 13 '18 at 15:02
  • 4
    $\begingroup$ By the way, regarding your intuition that it is impossible to evaluate neural networks on encrypted values and get the result in the clear, that's in fact feasible, using functional encryption. Existing constructions are horribly inefficient and purely of theoretical interest for any function class that would include deep neural networks, but for simpler function (e.g. evaluating degree 2 polynomials, which suffices for a wide range of statistics), it can be realised from fairly standard assumptions, in a relatively efficient way. $\endgroup$ – Geoffroy Couteau Feb 13 '18 at 15:06
  • 1
    $\begingroup$ Then I'm not sure I understood what you meant, doesn't the process of obtaining the result of evaluating a neural network in the clear on encrypted data qualifies as "learning" something from the encrypted data? In any case, you clearly stated that ' "learning on (..) encrypted data" seems to require at least some use of the decryption key.', which remains true - my point was rather that you can have something more fine-grained than 'no decryption key' or 'the full decryption key', by having a key that gives access to some non-trivial information (a function of the data), but not all of it. $\endgroup$ – Geoffroy Couteau Feb 23 '18 at 8:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.