If your secret material (you call this my secret
so I'll use that name) is chosen with sufficient min-entropy, then yes this scheme is acceptable. my secret
, however, must be in itself sufficiently hard to guess; it becomes effectively a password cracking problem. As such, I would suggest my secret
be a randomly generated keyfile, or at least a really secure password.
It's worth noting that if my secret
is found, this system completely fails. So guard my secret
very well. Be wary of keyloggers, shoulder surfers, malware, people nearby who happen to have a parabolic microphone (I'm not kidding), suspicious power outlets (also not kidding), and any number of other attack vectors.
Also, the benefit of a randomly generated key is, in the event of a compromise, you only need to change the key that gets compromised (or keys if multiple get compromised). In your deterministic case you have a single point of failure for the entire system, which according to Happy Quinn is not good (let's see if you can guess what I'm referencing).