Question about OpenSSL testing for FIPS 186-4 and using BN_generate_prime_ex

Question

I am working on testing my implementation of OpenSSL using the CAVP 186-4 RSA Validation System (RSA2VS).

There are tools under openssl-fips-2.0.16/test and I am working on modifying fips_rsagtest.c to process the 186-4 test vectors the way I need them.

The fips_rsagtest.c program was originally made to take in values for Xp1, Xp2, Xp, Xq1, Xq2, and Xq from an input file and generate the primes p and q.

But the newer tests require I generate the random values Xp1, Xp2, Xp, Xq1, Xq2, and Xq on my own. Looking at fips_rsagtest.c, it would create the primes p and q using the function RSA_X931_derive_ex().

I looked through these X931 functions and they do not appear to be the same functions OpenSSH calls: RSA_generate_key_ex().

My test lab has informed me that they have validated OpenSSL against 186-4 so I'm wondering if those implementations had custom code in them or if the built-in functionality in the current OpenSSL was able to pass the CAVP 186-4 tests.

That leads me to my 2 questions:

Q1) Does anyone know if it is OK to use these X931 functions for 186-4 testing?

Q2) If they are not, any pointers on how can I get these Xp1, Xp2, Xp, Xq1, Xq2, and Xq values out of the regular RSA_generate_key_ex() function? It appears to use BN_generate_prime_ex() but there are no obvious variables with the Xp1, Xp2, Xp, Xq1, Xq2, and Xq labels.

Thanks for any insight you can provide!

Answer 1

I believe I have answered my own question. It appears that OpenSSL uses FIPS 186-4 Appendix B.3.3 to produce its primes instead of Appendix B.3.6. The FIPS algorithms were using Appendix B.3.6 which is why I was out in left field.

Appendix B.3.3 does not use the Xp1, Xp2, Xp, Xq1, Xq2, and Xq values to produce the primes p and q.

I've started writing a test tool from scratch that isn't linked to the FIPS code instead of updating fips_rsagtest.c which just didn't want to work anyways (had issues getting the RBG to work in it).