I am coding a java based implementation of the NTRU public-key cryptosystem. I can comprehend the majority of the algorithms involved in the encryption and decryption process well enough, but the key generation process is giving me serious trouble.
I'll briefly go over some basics of the cryptosystem for those that are unfamiliar, otherwise, skip ahead to the paragraph titled The Problem.
We'll be working with polynomials with degree $<n$. Adding and subtracting these polynomials functions normally, however multiplying these polynomials works different¹². Given two polynomial $A$ and $B$ of degree $<n$, their product is $$A \cdot B = c_0\,x^0 + c_1\,x^1 + \ldots + c_{n-1}\,x^{n-1} = C$$ where each coefficient $c_k$ is calculated by³: $$c_k=\sum_{0\le i<n}a_i\,b_{(k-i\bmod n)}$$ multiplying each coefficient of $a$ with the same from $b$ in reverse order (if $b_k$ is the first coefficient in $b$ then $b_{k-1}$ loops around to the last coefficient in $b$ and so on) ensure the degree of the resulting polynomial remains $<n$.
The key $F$ is a polynomial with coefficients in $\{-1,0,1\}$. An example (where $n=7$) is: $$1\,x^0 + 0\,x^1 + 0\,x^2 + 1\,x^3 + -1\,x^4 + -1\,x^5 + 0\,x^6$$ or more plainly: $$1 + x^3 - x^4 - x^5$$
$F$ functions as the private key but must be verified to have inverse polynomials $F_q$ and $F_p$, which are polynomials of degree $<n$ with integer coefficients $q_i$ with $0 \leq q_i < q$, and $p_i$ with $0 \leq p_i < p$, where $q$ and $p$ are predefined integers ($q$ is coprime with prime $p$).
$F$ must satisfy that $F_q$ and $F_p$ exist given:
$$F \cdot F_q \equiv 1 \pmod q \quad\text{and}\quad F \cdot F_p \equiv 1 \pmod p$$ that is, if we define $G=F \cdot F_q$$C=F \cdot F_q$, its coefficients $g_i$$c_k$ must verify $g_0\bmod q=1$, and $g_i\bmod q=0$ when $1\le i<n$; and $$c_k\bmod q=\begin{cases}1&\text{if }k=0\\0&\text{otherwise}\end{cases}$$ and same for $F \cdot F_p \equiv 1 \pmod p$.
With all that being said…
The Problem
I am still struggling to grasp the algorithm used to calculate the polynomial inverses of the private polynomial key $F$, $F_p$, and $F_q$ for $$F\cdot F_q \equiv 1 \pmod q$$ and for $p$ etc.
Or to even verify if $F$ is invertible. I've seen different pseudocodes explaining the algorithm but all I've seen are poorly elaborated. Other explanations of the algorithm amount to "You can calculate the inverse using the extended euclidean algorithm" with no example, and looking at the eea myself I'm still none the wiser on how it's applied. I'd greatly appreciate a concise explanation in relation to the polynomial $F$ integer $p/q$ and polynomial degree $n$.
Let me know if there seems to be any key concepts I'm missing or critical variables I've omitted.
Notes of the editors:
¹ In this modified polynomial multiplication, $x^n$ is assumed to be $1$ in coefficients of degree $\ge n$.
² Equivalently, this is polynomial multiplication modulo $x^n-1$.
³ $u \bmod n$ is defined as the integer $v$ with $0\le v<n$ and $u−v$ a multiple of $n$.