It seems that RSA-KEM has a very troublesome method of generating the secret $r$. It seems that the random value needs to be in the range $0 \le r \le {n - 1}$.
Now most cryptographic environments don't have a random number generator that generates those kind of numbers; usually they are just provided with a seeded DRBG that outputs full bytes. Performing all kinds of integer operations on the bytes may lead to leaks of the master key within these kind of environments. Especially the $\bmod$ operation seems to be tricky in this regards.
Is there a known good way of generating $r$ that disposes of the $\bmod$ operation and possible looping?
For instance I could think of constructing $r$ as
00 | R
The |
operation is of course concatenation.
In this construction $R$ is then an octet string consisting of (pseudo) random bytes. $r$ consists of the value of 00 | R
when used as a big endian unsigned integer. It can be used directly as input to raw "RSA encryption" as implemented by most libraries.
That would provide $\operatorname{len}(R) * 8$ bits of entropy for the KDF. I think that would be $\lfloor (log_2(n) - 1) / 8\rfloor \cdot 8$ bits of entropy when measured against the modulus value $n$.
The scheme would otherwise be unaltered of course; there would be no explicit checks on the value of $r$ during decryption.
Would this suffice as a secure scheme of constructing $r$ instead of performing integer calculations and checks to get to $0 \le r \le {n - 1}$?
(data && mask) || floor
, where mask is equal to(1 << required_bits) - 1
and floor is set to the minimum bound to retrieve a value x: floor <= x <= mask. This appears to provide me with a value in the appropriate range in constant time, but I am not certain of how "clean" it is in terms of bias or if it's helpful to you here (I see you need x < y <z whereas I need x <= y <= z). $\endgroup$