It seems we have aligned interests. I'm also a university student (although I am a math/comp sci double major) looking to pursue a career in cryptography. To that end, I have been self-studying it for a while now. So, take what I say with a grain of salt. As a forewarning, this post focuses specifically on what topics cryptographers often encounter and less about what, as a person, makes you useful as an employee to a crypto-agency.
From what I can best tell, the requisite knowledge of computer science is entirely dependent on what you want to do with cryptography exactly. If you are looking to work in an applied area, then you had best had a strong grasp of the basics of computer science. If you are implementing algorithms, then you will likely need to know the basics of modern-day computer architecture.
The reasons for this are many-fold. First, an understanding of computer architecture will allow you to optimize code like no other. For instance, a knowledge of things like instruction-level parallelism and pipelining is incredibly important because taking proper advantage of both can seriously speed code up. Second, you may need to know some basics about architecture in order to ensure security of your scheme. For instance, if you are working with smart cards, you will need to be able to design, or at least verify the security of, tamper-resistant circuitry.
Further, you will need a strong knowledge of C, which sits so close to the hardware that it isn't too far a step away anyways. From what I can tell, the vast majority of "serious" crypto code is written in C, even if it has a language wrapper around it (e.g. PyCrypto). Other languages, especially interpreted ones, simply do not allow the same performance as C, and we do like performant implementations. Another selling point of C is that, in crypto, you are most often working with raw bytes and bits, which is exactly where C shines. It's the de facto language for low-level work.
To that end, I would really recommend you pick up a minor in computer science if you are going to work with applied cryptography at all. Relevant courses include introductory programming, operating systems, assembly language, architecture, and complexity theory. There are lots of computer science resources on the web, though, so don't feel too discourage if a minor is simply not possible! An excellent C resource is Learn C the Hard Way. If you want to know more about operating systems (especially important if the code you write will be run on a real-life computer...) then Operating Systems Design and Implementation is an excellent resource. Unfortunately, I don't know of a good reference on computer architecture because of the extreme fast-paced movement in the field; I recommend trying to understand some of the optimizations current crypto algorithms implement.
Of course, to get very far in crypto, you will have to have a strong understanding of mathematics. The Handbook of Applied Cryptography, an excellent reference for crypto in general (with the entire book free online, by the way, on the linked site), has an entire chapter devoted solely to the mathematical background necessary, and then another entire chapter just for number theoretic problems. Since you are a math student, this is probably where you shine.
The HAC lists probability theory, information theory, complexity theory, number theory, and abstract algebra as being introductory background material, and the rabbit hole just goes deeper and deeper. In that list, I don't think there is any particular area that is neglected; all of them are used frequently in different areas of crypto. I can give specific examples if you would like.
You will also need the tools of theoretical computer science. (Sometimes, you will even see cryptography as a field under theoretical computer science, though I feel this definition is rather restrictive.)
Of course, complexity theory is already on the list, and it is indeed the premier star of the show as far as TCS topics in cryptography go. But you might also need to know a few models of computation, e.g. Boolean circuits, Turing machines, random access machines, and the like, because sometimes they're important to a particular problem. (For instance, Craig Gentry used Boolean circuits as the computational model in his fully homomorphic encryption scheme presented in his thesis.) You might consider these models of computation a part of computational complexity theory, though.
Information theory is also on the list of topics in the HAC, and you might find it sometimes included in theoretical computer science lists. Honestly, the line between theoretical computer science and math is often blurred.
For good references in theoretical computer science, I would first recommend a good text on complexity theory; the one I studied was Computational Complexity by Papadimitriou. The wonderful folks over at cstheory.se recommend two other works on complexity theory, but I can offer no evaluation of either. They also have a post devoted to some good books for crypto, which I just now spotted! There are some on the list there I haven't seen. The full post is, in general, worth checking out, but I thought I would link you to crypto-specific areas.
Of course, there are areas of cryptography that are, shall we say, less often "applied" to the real world than others. An example of such is the idea of a zero-knowledge proof, namely proving you know something without revealing what that something is. These things are applied sometimes, and their applications are fantastic, but they haven't been --- as far as I can tell --- widely adopted as, say, generic "guarantee confidentiality and integrity" areas of crypto (e.g. things like AES+HMAC). At least, not in the public eye. If you are not convinced, take a look at the front page of this StackExchange.
For a good reference on "theoretical" crypto topics, I would highly recommend Foundations of Cryptography by Goldreich. It's a two-volume work and covers the real theoretical material lurking in the background. The second book is more applied than the first. Further, Katz and Lindell's Introduction to Modern Cryptography is also quite good, although I haven't studied it extensively.
Of course, there are also generic crypto references, which is probably what you're most after here. Applied Cryptography and other books by Schneier (et al?) are good. The HAC, which I linked to above, is an excellent tome, touching on tons of topics in applied crypto with a heavy dose of theory too. The post I linked to above (on cstheory.se) has a few recommendations too. For a good history of the subject matter, I would highly recommend (if you want brevity) The Code Book, which is a really nice overview of the history of crypto. Of course, if that interests you, there is always the tome on the subject, Kahn's The Codebreakers. You can also see Thomas Pornin's excellent answer on a question similar to yours on security.se.
In short, this post is a high-level overview of some possible self-study options in crypto. Since I am only a university student, I can't tell you what employers do and don't want you know (especially not the NSA), nor can I tell you what topics are the most important in current-day research. I also make no claims that this post is complete and doesn't make glaring omissions. But I think it has been fairly accurate, at least to the extent that my knowledge and experience permit.
