What you could do, is use a Mill's constant for generation. Then, a test for primarity would be good anyways...(in case that not exact enough constant is chosen, so you could end up with non prime)
Mill's constant is such a number, that if powered to 3, and then powered to any N, where N is an usigned integer, we get value V wich, rounded down, is a prime.
So, let C be constant
C = 1.3063778838630806904686144926 ....
L = C^3
Then choose any N from <0, infinite> , N is whole
prime = round down( L^N )
It has several drawbacks, such as the constant must be very accurate, if you do not have accurate enough constant, you will end up with composed number. However, it is proven that this ,,algorithm" always generates prime numbers.
Next drawback I heard about is that as you increase the exponent, more computation power will be needed. However, all you need to do is generate two of them, so it could do...
For further reading, here is what I found as reference.