This might be a stupid question, so bear with me. I was wondering if LLM embeddings can be used to anonymize input text. I couldn't find any information online that says that embeddings can be 1:1 decoded back to the original text.

An example: A user wants to check some metadata about a query with an external API, but doesn't want the exact text input to be sent to the API (it might contain personal/sensitive information).

Can he send the embeddings to the API, assuming the API can do meaningful checks on the embeddings - so the API will not have access to the original text?

  • $\begingroup$ LLM stands for Large Language Models; ChatGPT uses one. LLM embeddings are the data structures used by LLMs to encode the information initially fed during a training phase. LLM embeddings are the output of training, and an input of later phases. So the question proposes to train a LLM with input text, and send the data produced by the training to the API (it's not proposing to make the API submit queries to the trained LLM, which could be another approach). $\endgroup$
    – fgrieu
    Jun 3, 2023 at 10:25
  • $\begingroup$ @fgrieu, Yes, so for example I would want to anonymise "yes, my name is John Doe (secret)" and so the API could understand that the message was a "yes" and that a name was provided but not the exact name "John Doe". $\endgroup$
    – funerr
    Jun 3, 2023 at 11:39

1 Answer 1


This is an interesting question! However, it requires being a bit more formal about what "anonymization" means to answer properly. No approach will result in a leakage-free solution, since some information will always be revealed about the input unless you use encryption or secret sharing to hide the original input.

As you observed, embeddings are not one-to-one, which means some information will indeed be "lost" during the embedding phase. The problem is then to determine how much information is lost exactly, and making sure it is sufficient to hide sensitive information in your application/setting. Two possible approaches come to mind:

(Local) Differential Privacy

To ensure that each embedding always loses "enough" information (here enough is a setting-specific parameter) to prevent the API from learning sensitive information about the query, we can inject some noise to the embedding and analyze the information loss through the local differential privacy framework. Interestingly, when embedding high-dimensional vectors into a low(er)-dimensional space via the Johnson–Lindenstrauss transform, it has been shown that the embedding itself can be differentially private under the right parameters.

Probably Approximately Correct (PAC) Security

A different approach (compared to local differential privacy) was recently proposed to capture the information leaked on an input $X$ from a mechanism $M$ applied on $X$. In your case, $M$ would be the embedding and $X$ would be the query. PAC security quantifies the leakage in a data-dependent way (whereas differential privacy considers worst-case data distributions). This appears to be closer to your goal of determining how "private" the embedding is. I recommend reading Section 7 and Appendix K of the PAC security paper to get an overview and comparison to differential privacy.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.