Why should one model an entropy source in order to build a TRNG?
I sell two black boxes with a crank and an output display on it in my cryptography shop. You can pick one to take home for use in cryptography.
- One box contains a gremlin who dutifully computes the bits of $\operatorname{AES}_0(0) \mathbin\| \operatorname{AES}_0(1) \mathbin\| \operatorname{AES}_0(2) \mathbin\| \cdots$ to display on the output when you turn the crank.
- One box contains a gremlin who tosses a coin and displays the outcome when you turn the crank.
The boxes are labeled AES GREMLIN and COIN-FLIPPING GREMLIN, so the adversary, who is watching the shop through the window, knows which box you picked. You, however, deliberately ignore the labels, pick one arbitrarily, and feed the output to a generic statistical test like the NIST suite. It returns a happy result in either case, so you take it and go on your merry way.
If you took the AES GREMLIN box, you would lose against the adversary no matter what fancy cryptography you use it to generate keys for. If you took the COIN-FLIPPING GREMLIN box, well, whether you win against the adversary depends on what cryptography you use it for—and only on what cryptography you use it for, because this gremlin is as unpredictable as they get.
This is why you should pay attention to the information you have about the world, instead of deliberately ignoring it!
- What is a "model"? For example, are we talking a simple histogram of the raw output distribution, or something like $ \sigma^2_{quantum} = \frac{\gamma}{\gamma + 1} \langle V(t)^2 \rangle $ thus $ \sim H_{Shannon} $ in laser phase interference?
A model is a description of a probability distribution on outputs that represents your state of knowledge about what they might be. In the AES GREMLIN model, after a short computation anyone knows exactly with no uncertainty what the output will be. In the COIN-FLIPPING GREMLIN model, every possible outcome is equally probable: nobody has any reason to suspect that the next output will be 0 rather than 1 or vice versa. It might be a family of models with parameters, or it might be a composite model with Bayesian model selection, etc.
Other models include:
- Counting the number of ionizing events in a duration of one second from a radiation source. In a short period of time, each such number will have Poisson distribution for some fixed rate $\lambda$ depending on the radiation source. Over time, the radiation source will decay, so the Poisson rate $\lambda_t$ at time $t$ will dwindle—fast, if it has a short half-life; slowly, if not.
- Reporting 0 or 1 depending on which side of a beam-splitting polarizer a photon from an unpolarized single-photon emitter passed through. The probability of 0 or 1 from each sample depends on the structure and aberrations of the beam splitter, and on the characteristics of the emitter. If the cat disturbs the setup by jumping on the experiment table and pointing the emitter away from the splitter, the probability of 0 might go to 100%.
- Is there any original theoretical work justifying this modelling requirement? I'm asking about any works underpinning what one might read in a FIPS/BSI AIS document.
First principles.
The more you know about the physical device, the better you can make predictions about it, like the half-life and purity of the radiation source, or the aberrations of the beam splitter and distribution of polarizations of the emitter. If you plug up your ears and cover your eyes, that doesn't make it less predictable to an intelligent adversary!
Your job, as a designer or implementer or cryptographer, is to do the best you possibly can with the state of the art and physics and engineering to predict the outcome of the device. Then an adversary—who is at least as good at physics as you are, and who is smart enough not to ignore relevant information—might not have a much better chance at predicting it.
Further, as an engineer, you know systems break down: the uranium decays, the cat jumps on the table and knocks the beam splitter over, the silicon crystals degrade under stress of avalanche breakdown, the adversary shines a bright light on your photon detector, etc. Knowledge of the engineering enables you to predict plausible failure modes, and write tests that have a high probability of raising an alarm in all the failure modes, but low probability of false alarms when the device is working as intended.
- What to do if the empirically measured entropy rate does not agree with that predicted by said model? Say an error of >50%.
Background: Every model has a definite entropy. Sometimes families of models are related. For example, the following describes many related models in terms of a parameter $p$:
- A box containing a gremlin who makes independent but biased coin tosses, coming up heads with probability $p$, and displays the outcomes, when you turn the crank.
The min-entropy of a single outcome from this model is $-\log_2 p$. Generic ‘entropy estimators’ posit really simple families of models like this; empirically guess the value of $p$ based on samples of data; and then analytically compute the entropy of the model with their guess for $p$.
Suppose you compute the entropy for your model involving a Rube Goldberg machine of a radiation source, a photon emitter, a beam splitter, an avalanche diode, and a parrot, based on your understanding of the physics and engineering and ornithology of the system. It doesn't matter that this system give the highest possible entropy per bit of output; what matters is that you do a good job computing what the entropy per bit of output is, even if it's only 0.1 bits of entropy per bit of output.
Suppose a stupid entropy estimator, which was designed without knowledge of your system because someone at NIST wrote it a decade before you even met the parrot, guesses lower entropy than you computed. This suggests that a stupid adversary, who doesn't even know how the device works, can do a better job at predicting the output than you can.
What this means is that you are bad at physicsing, and you should try harder.
There is now a hot network question - Is a model fitted to data or is data fitted to a model? Currently answers are leaning towards the empirical data taking priority over the theory. So you'd start with the oscilloscope and work upstream to the physicsing[sic]. As a philosophical empiricist, this seems more intuitive to me but there does seem to be a portion of members here that give precedence to math over measurement. It's a Plato v Aristotle question and I'm trying to be open minded on this one.
The HNQ is about a rather mundane question of phrasing, not of what the phrase means; there's nothing profound there. The asker found an article from Wolfram that described the standard process of using a set of data to find parameters in a family of models (like estimating $p$ in the above $p$-weighted gremlin); the Wolfram article said ‘fitting data to a model’ even though the observed data are fixed and the model parameters are variable, while other articles say ‘fitting models to data’; the asker asked about the discrepancy in phrasing.
Obviously when studying an ill-understood physical system to learn about it, a sensible scientist will consider many possible models and use tests to decide which ones to prefer, and hypothesize new ones on the basis of patterns seen in empirical observations.
This question is not about how to science your way into a grand unified theory of quantum gravity. There is arbitrarily much to be said about the practical and philosophical underpinnings of empirical reasoning and the formal frameworks for doing it, like frequentist statistics and hypothesis testing or Bayesian inference—of which some has been said already, but the broader topic of which is far outside the scope of crypto.se.
Rather, this question is about how to build a TRNG. For that, there is plenty of already well-understood science out there which you can use as building blocks. If you consciously ignore it all, and sit on the laurels of a dieharder run, you're doing a bad job of engineering. Will the radiation source decay? Will the silicon degrade? Will temperature, pressure, and humidity affect the result? Dieharder can't tell you about any of these.
