A colleague gave me the following explanation that I think makes a lot of intuitive sense, so I'm reproducing it here. Skip to the last paragraph it you don't care about the proof.
Suppose you're trying to track one individual user, and you're trying to figure out whether they're in the database. You have some prior knowledge about this: $\frac{P(in)}{P(out)}$ is how much more likely you think the user is in the database than isn't.
How does this change once you know the output $O$ of the algorithm? The posterior knowlege can be written as $\frac{P(in|O)}{P(out|O)}$. We can use Bayes' theorem to decompose it:
$$\frac{P(in|O)}{P(out|O)}=\frac{P(in)P(O|in)}{P(out)P(O|out)}=\frac{P(in)}{P(out)}\cdot \frac{P(O|in)}{P(O|out)}$$
If the algorithm is $\varepsilon$-differentially private, then it satisfies $e^{-\varepsilon}\le\frac{P(O|in)}{P(O|out)}\le e^{\varepsilon}$, since there's only one user different between the two scenarios. Using the previous formula, we get:
$$
e^{-\varepsilon} \cdot \frac{P(in)}{P(out)} \le
\frac{P(in|O)}{P(out|O)}
\le e^{\varepsilon} \cdot \frac{P(in)}{P(out)}
$$
This is essentially telling us that $e^\varepsilon$ is how much confidence we can gain about whether one user has been added/removed from the data, by looking at the output. When $\varepsilon=\log(2)$, we can gain a factor of $2$ in the knowledge we have: if we had no information at the beginning (say, $P(in)=P(out)=0.5$), now we might have information like $P(in|O)=2/3$ and $P(out|O)=1/3$, but not a bigger difference.
If you want to dive a bit deeper, I wrote a blog post to elaborate a bit on this reasoning and give a concrete example. I also explain how the formula above can be used to find bounds on the possible values of $P(in|O)$ depending on $P(in)$ and $\varepsilon$:
