First, affine transformations $S,T$ are defined by $S=A_1+v_s, T=A_2+v_t$. Let the private polynomial function $\mathcal{F}$ be known. The short description of the public key map is $P(X) = T \circ \mathcal{F} \circ S(X)$. Recall that common multivariate schemes are based on the security assumption of IP2, this is, the Isomorphism Of Polynomials with two secrets.
It results that only applying transformations $T$ and $S=I_n$ (the identity) makes the scheme vulnerable by linear algebra attacks. We can recover the transformation matrix $T$ since we already know that it transformed $\mathcal{F}$ to the public key matrix.
In addition, using $T=I_n$ and $S$ an arbitrary invertible affine function, the problem now is equivalent to IP1, which can be stated from the so called Polynomial Affine Equivalence problem (PAE) (see [1] section $2.3$ and $3$). Given $\mathcal{F}(x)$ and $\mathcal{F}(S(x))$ find $S$. Schemes based on IP1 can be vulnerable to attacks if not designed with care, see [2].
Then we are restricted to IP2 if we want to mantain security from this point. Recall that including abritrary invertible affine transformations $S,T$ would make the input-output of $\mathcal{F}$ unknown, even knowing $\mathcal{F}$. Then as $T$ transforms an unknown input, we learn nothing from here nor from the view of PAE or IP1, as we only know one set of quadratic equations $B$ but the set $A$ remains hidden by the transformation $T$.
In the previous description I tried to demonstrate that the knowledge of the private polynomial $\mathcal{F}$ has no effect when the scheme is based on the assumptions of IP2.
However, is there any complication if the private polynomial $\mathcal{F}$ is publicly known from other perspective?
My intiution says that recovering $T$ is fatal as $\mathcal{F}(X), \mathcal{F^{-1}}(X)$ are known the attacker obtains $F^{-1}(Y)=S(X)$. Then to solve IP1 he must establish an affine equivalence function $S$ between $\mathcal{F}(X)$ and $\mathcal{F}(S(X))$
If the attacker doesn't possess $\mathcal{F}$ knowing $T$ will yield $F \circ S(X)$ but yet he doesn't know $\mathcal{F}(X)$, thus IP1 cannot be followed from here.