Out of the $10001$ possible outputs
- $7295$ occur with a probability exactly equal to $429497$ in $2^{32}$.
- $2705$ occur with a probability exactly equal to $429496$ in $2^{32}$.
- One value, specifically the value ten thousand, occurs with probability exactly equal to $1$ in $2^{32}$.
The mean value of the distribution is approximately $4999.5$. It is approximately a uniform distribution over the range $[0, 9999]$. It is much more uniform than a real-world fair die roll's distribution, but it's also a huge bias in the world of cryptography.
It's not a bias large enough to discourage gamblers. Cryptographers aren't going to be inclined to gamble even for more fair odds based only on the odds and payouts (A lot like the story/rumor about physicists' conferences being unwelcome in Vegas. However that's not to say mathematically minded people might still be irrational or find other reasons for playing.)
Ignoring the value $10000$, each of the $7295$ over-represented values deviates from its expected probability by a factor of $1.0000006298$ and each of the other $2705$ values deviates by a factor of $0.9999983015$.
The multiplication method (using exact math, floating-point arithmetic is not necessarily exact) of transforming one uniform distribution to another uniform distribution over a smaller range has the same amount of bias as the naive division-remainder (without rejection sampling) method that people warn programmers not to use. The bias for the multiplication method, however, is more subtle because it doesn't lead to the over-represented elements all being low elements.
For this specific distribution $0$, $1$, and $2$ are all slightly over-represented. $3$ is slightly under-represented. $4$, $5$, $6$ over-represented, $7$ under-represented, $8$, $9$, $10$ over-represented, $11$ and $12$ over-represented, $13$ under-represented. I think most (or all) of the over-represented elements occur in runs of two or three. And the under-represented in runs of one.
As for the protocol you described, I can't say much about the specifics because you aren't precise in your description.
I can say that the output of algorithms like SHA-512, and by extension HMAC-SHA-512, can safely by treated as a uniform distribution of all 512-bit long bit-strings. Hash functions are often modeled as a random oracle. Wikipedia describes it as:
In cryptography, a random oracle is an oracle (a theoretical black box) that responds to every unique query with a (truly) random response chosen uniformly from its output domain. If a query is repeated it responds the same way every time that query is submitted.
The hash output can begin with 00000000
or FFFFFFFF
or any other 32-bit value. In fact, every possible prefix of a given length $n$ is expected to occur each with the probability $2^{-n}$.
It is possible to create a fair shared random value between multiple parties using hash functions. Two parties choose a random secret number, exchange those numbers between each other simultaneously (so no one can cheat by changing their number), and create a new agreed-upon shared random value by hashing the two secrets. (They obviously need to agree on the order they hash each number.)
A commitment scheme is used to prevent parties from changing their secret number after observing the other player's number. They send each other hashes of their own secrets. They don't reveal to each other the actual value of their secret by exchanging hashes. It's not possible to determine the input from just the hash function's output assuming the input is sufficiently unpredictable. (Having high entropy.) A player cannot swap out the secret value they commit to without finding a collision in the hash function. (Using a prepended random nonce makes finding collisions even harder because you cannot use precomputed collisions.)
The final hash of both secrets, the nonce, and whatever other data is involved results in a random value from a uniform distribution (under the assumption that the hash function behaves like a random oracle). Any change in input to a hash function results in unpredictable changes to the output. This means that one party doesn't have to trust that the other party's secret is random as long as their own number was chosen randomly.
However, even with commitment schemes, it is still possible to cheat if aborting the protocol allows the second person to walk away with a smaller loss by not cooperating by revealing their secret than the loss they would incur by cooperating. (Sort of like people who hit the reset button on their game consoles when they're losing to another player. Or flipping a chess board off the table.) The second person has enough information to calculate the result of the game on their own, and so they might pretend to have their computer crash to avoid paying out larger winnings.
And again, your description of the protocol is vague, so we can't say whether the actual protocol is secure or not. (And even if the protocol were secure it's another matter whether or not the implementation is correct.)