# Calculating the privacy that Renyi ($\epsilon, \delta$) Differential Privacy satisfies

I add differential privacy (DP) to my machine learning models by using PyTorch-DP. PyTorch-DP supplies me with the values: $$\epsilon$$ and $$\delta$$. I know that the $$\epsilon$$ tells us something about the probability $$\rho$$ that the privacy of individuals in the dataset (DP) is broken. So, a $$\rho$$ of 0.5 might correspond to an $$\epsilon$$ of 2 (depending on the statistics that are released).

So now, my question is: With the implementation that PyTorch-DP uses (Renyi DP calculated by using the Moments Accountant method), to what $$\rho$$ does the $$\epsilon$$ correspond to? If I do not know this, how can I say something "meaningful" about the level of privacy $$\rho$$ that $$\epsilon$$ satisfies?

• This is cross-posted on Information Security site. Please if you are the same person, delete one copy. Jun 24, 2020 at 14:25
• I don't think your interpretation of $\varepsilon$ is quite right. What do you mean by "the probability $\rho$ that the privacy of individuals in the dataset is broken"? I generally find the Bayesian interpretation of $\varepsilon$ to be the most accessible. I also don't understand how this question is about Rényi privacy, whose parameters are not $(\varepsilon,\delta)$ but $(\alpha,\varepsilon)$. Can you clarify?
– Ted
Sep 25, 2020 at 12:17

I don't think your interpretation of $$\varepsilon$$ in DP is correct. You can find a decent explanation of this parameter in this crypto.SE question and links therein.
The best intuition I have for Rényi privacy is that it bounds the average of the privacy loss) $$L(O)$$: instead of being a worst-case property, like DP ("for all outputs $$O$$, $$L(O)$$ is lower than $$\varepsilon$$"), Rényi privacy is an average-case property ("when we average $$L(O)$$ across all possible outputs $$O$$, the result is lower than $$\varepsilon$$"). The $$\alpha$$ parameter determines which kind of average we're using: used: $$\alpha=1$$ bounds the arithmetic mean of $$L(O)$$, $$\alpha=2$$ bounds the arithmetic mean of $$e^{L(O)}$$, $$\alpha=3$$ bounds the quadratic mean of $$e^L(O)$$, etc.