The SRP paper has this point in its list of security properties:
6. If the user's password itself is compromised, it should not allow the intruder to determine the session key K for past sessions and decrypt them. Even present sessions should at least be protected from passive eavesdropping.
The following section is titled Reduction to Diffie-Hellman and shows that, given an algorithm which retrieves the session key from the transmitted values and the password, you can also solve the computational Diffie-Hellman problem.
So yes, SRP is as secure as Diffie-Hellman for key exchange if the password is known to the attacker (and the other parameters are chosen properly, i.e. uniform random in their respective domains).
On the other hand, SRP with a non-secret password has the same problem as the pure Diffie-Hellman key exchange, when facing an active attack, i.e. a man-in-the-middle or an attacker who impersonates only one of the two parties.
I would like to do key exchange under the assumption that I've already
done mutual authentication (through some out of band user intervention).
Sorry, this rings alarm bells. Depending on how this out of band user intervention looks like, it could prove nothing at all about your actual communication going on, or about the non-existence of a man-in-the-middle attack.
Your "out of band authentication" should at least verify some value derived from the negotiated secret $K$, or (if you want it to go on beforehand) negotiate some (real) secret which then goes into the process as a part of the "password", so the attacker actually doesn't know the password, and can't mount a man-in-the-middle attack.
Also, as mentioned by CodesInChaos in a comment, SRP (as DH) does a lot of exponentiating of values by secret exponents. This is prone to timing and power analysis attacks if done naively, so your "default" JavaScript BigInteger implementation will have problems here.
If you want to run this in a Web browser (what JavaScript leads to believe), also take note that this will not more secure than the connection by which the browser got the code (hopefully at least an HTTPS connection), and good random numbers are a critical problem there. Also, side channel attacks are even easier if run on an untrusted platform like a web browser. More details in the article Javascript Cryptography Considered Harmful.