On modern CPUs, a fast Cryptographically Secure Pseudo-Random Number Generator runs sizably faster than one cycle per byte. We are talking >40Gbit/s. See numbers [there][1]. Top contenders are [AES][2]-[CTR][3] assisted by special instruction, and [ARX][4] ciphers like [ChaCha][5].

When using dedicated hardware, the true limit is moving around the generated random bits. We can arbitrarily parallelize CSPRNGs, and we have many efficient designs. For example, [Trivium][6] is [reported][7] at >120Gbit/s/mm<sup>2</sup> using 250nm metal CMOS technology, and state of the art today is about 10nm technology, which is faster and some hundreds time denser.
___
Update per [comment][8]: All modern CSPRNGs require $O(n)$ bit operations to produce $n$ bits, and we want a more precise estimate. [ChaCha][5] with $r$ rounds (see [source][9]) produces $16$ words each 32-bit with computational cost dominated by $16(r+1)$ 32-bit additions and $16r$ 32-bit XORs. Using NAND gates, a [ripple-carry adder][10] is 9 gates, XOR is 4 gates, thus the cost is $13r+9$ NAND bit operations per bit produced. For ChaCha8 (which is hoped to have at least 128-bit security), the cost is $\approx113n$ NAND bit operations to produce $n$ bits (with no consideration for circuit depth).
___
Update on the security of ChaCha8 per this [other comment][11]: I'm reasonably confident that ChaCha8 is at least 128-bit safe, but reluctant to state much more because

- ChaCha is an evolution of Salsa20, with the same number of 32-bit additions and XORs, and a faster diffusion. While that could make it safer, I'm using that as safety margin.
- In the abstract of the [Salsa paper][12], Daniel J. Bernstein recommends the 20-round version for typical applications, and presents 12 and 8-round variants as _reduced-round_ versions recommended _for applications where speed is more important than confidence_.
- Salsa20 was proposed for eSTREAM, in the category asking for fast software ciphers with 128-bit security. While DJB presents Salsa20/20, Salsa20/12 and Salsa20/8 as 256-bit ciphers (which they are as far as key size goes), I'm not reading him as claiming 256-bit security, especially for the reduced-round versions.
- There is a known attack with complexity $\approx2^{249}$ against Salsa20/8, thus definitely it does not have security matching its key size. Again, ChaCha8 differs only by its diffusion pattern.
- There is a known attack with complexity $\approx2^{153}$ against Salsa20/7, and as the saying goes: attacks only get better; they never get worse.


  [1]: https://bench.cr.yp.to/results-stream.html
  [2]: https://en.wikipedia.org/wiki/Advanced_Encryption_Standard
  [3]: https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#CTR
  [4]: https://en.wikipedia.org/wiki/Block_cipher#ARX_(add%E2%80%93rotate%E2%80%93xor)
  [5]: https://cr.yp.to/chacha.html
  [6]: https://en.wikipedia.org/wiki/Trivium_(cipher)
  [7]: http://www.ecrypt.eu.org/stream/e2-trivium.html
  [8]: https://crypto.stackexchange.com/posts/comments/138736
  [9]: https://crypto.stackexchange.com/q/26437/555
  [10]: https://en.wikipedia.org/wiki/Adder_(electronics)#Ripple-carry_adder
  [11]: https://crypto.stackexchange.com/posts/comments/138761?noredirect=1
  [12]: https://cr.yp.to/snuffle/salsafamily-20071225.pdf