This is a soft question regarding comparisons between government security services (eg, NSA or GCHQ) and open-source research (e.g., academia). Hopefully it's on-topic for this site!
In essence, my question is the following.
How far ahead (if at all) of of open source (specifically academic) research are governmental agencies?
There are some various thoughts I've had on this matter.
NSA is the largest employer of mathematicians worldwide, and their primary focus is on security. So not only do they have an incredibly large number of people, but they're also very focused. On the other hand, cryptography departments at universities (that I'm aware of) don't seem to be particularly large.
We now get onto the actual people who do the work. For example, to start at the base level at GCHQ you need a 2:1 in a three-year maths degree. (I'm from the UK, and don't really know any comparison with American GPA.) In comparison with university faculties, these will all have doctorates as a minimum, and university researcher places (at least in UK universities I'm aware of) are very competitive.
At a first glance, this suggests that the standard of researchers in general is much higher in academia. Of course, one has to be careful: not everyone wants to do a PhD for starters; moreover, a lot of people may not want the job-insecurity of post-docs, and prefer to move into a more secure (likely higher-paying) job outside of academia. Furthermore, continuing from the first point, these people then spend all their time working on this stuff, with large amounts of resources (including many other people around).
As an example, when DES was replaced with AES, instead of developing AES themselves, the NIST set a challenge to other researchers. The NIST then chose the best of these (supposedly, anyway; many people have apparently legitimate claims that the most secure wasn't chosen, but instead the one the the NIST 'liked' the best).
I have heard that RSA was discovered some 8 (or maybe 3?) years before it was public, but that this was at a time when there was comparatively little open source research into such topics.
Finally, a lot of what secret agencies do is, well, secret!
Basically, there seem to be arguments for many points, and I was wondering if anyone has more insight than I do!
Very related (but not the same) is this question: How can we reason about the cryptographic capabilities of code-breaking agencies like the NSA or GCHQ?; that question is more about some specific things (eg symmetric ciphers).
To be clear, I know this question isn't definitively answerable without high-up knowledge of governmental affairs, but I feel some approximations should be possible.
Of course, none of this addresses private companies (eg, Google). These employ a lot of people, likely(?) with higher entry requirements than NSA/GCHQ (see above on why that doesn't give a clear cut implication) and also have lots of money; moreover, there's likely(?) less bureaucracy.
If any of my above claims are wrong, please do point them out! I'm not very knowledgeable in the world of cryptography. I'm a maths PhD (discrete probability), but have interest in this stuff.