I must be missing something, and I am sorry if this is a novice question...
I have seen in places (eg. https://crypto.stanford.edu/~dwu4/talks/SecurityLunch0214.pdf page 13) that linear regression is possible on the cipher text of a fully Homomorphic Encryption system.
My question is: if you can generate the inverse of the matrix (X^T X), then can't you generate the cipher version of the identity matrix, and determine what E(1) is? And if you have E(1) can't you generate E(2) ... E(n) using either encrypted addition or plain text multiplication?