I think that you're asking how to generate a timestamp response as defined in timestamp-protocol: RFC3161, with
openssl to generate and sign the response using a
PKCS#11 (HSM in your case) as a TSA signer.
I think that there is no native way to use
openssl to do this. (maybe with some plugin like: opensc pkcs11 engine for openssl).
If you take a look on
ts command documentation: https://www.openssl.org/docs/apps/ts.html. To generate a timestamp response you can use the follow command:
openssl ts -reply [-config configfile] [-section tsa_section] [-queryfile request.tsq] [-passin password_src] [-signer tsa_cert.pem] [-inkey private.pem] [-chain certs_file.pem] [-policy object_id] [-in response.tsr] [-token_in] [-out response.tsr] [-token_out] [-text] [-engine id]
The TSA signer certificate and private key are specified in the following parameters which specify that both must be in PEM format, so seems that use of
PKCS#11 it's not possible.
The signer certificate of the TSA in PEM format. The TSA signing certificate must have exactly one extended key usage assigned to it: timeStamping. The extended key usage must also be critical, otherwise the certificate is going to be refused. Overrides the signer_cert variable of the config file. (Optional)
The signer private key of the TSA in PEM format. Overrides the signer_key config file option. (Optional)
However as I comment before, it can be possible maybe using opensc pkcs11 engine for openssl, I can't confirm if it works because I'm unfamiliar with it. Take a look on this quickstart guide which may put you on the right direction.
Hope this helps,