You could try portecle. Import your
keystore.p12 store, using the correct password of course or it will (silently) fail. Delete the certificate entry, and import a new "trusted" certificate entry.
Now it could be that some PKCS#12 defined attributes are different (e.g. friendlyName, which is visible as
alias in portecle. Normally the key and certificate are linked through the modulus value however, so this should not make a difference for anybody trying to use the private key and certificate. It's probably best to copy the
alias of the certificate just to be sure.
Portecle is written to use the Java
KeyStore class, which supports PKCS#12 out of the box. The Java
keytool utility uses the same class, but it seems to automatically delete certificates and keys together so it is not much use here.