Does order matter for RSA when used multiple times?

I actually have no idea how RSA works, but I currently (think that I) know

1. Public and private keys are mathematically doing their opposites
2. Therefore, the order in which they are applied doesn't matter
3. The system is cyclic and applying one key repeated times will eventually lead to the starting number

Hoping I am OK till here, these made we wonder if you can apply the keys in any arbitrary order multiple times and come back to the starting number, as long as you use both keys the same number of times.

• Welcome to crypto.SE, packet0. – Maarten Bodewes Sep 24 '16 at 16:36
• ...and welcome to the Hot Network Questions list, too. :) – Ilmari Karonen Sep 24 '16 at 18:25
• Thanks. I won't be going on the list so often probably :p – packet0 Sep 25 '16 at 3:24

In textbook RSA, one can apply the public key function $x\to E(x) =x^e\bmod n$ and the private key function $x\to D(x)=x^d\bmod n$ in any arbitrary order multiple times and come back to the starting number, as long as one use both keys the same number of times, and the starting number was in the set $\{0,1,\dots,n-1\}$ (also known as $\mathbb Z_n$ ).

Proof follows from the fact that, assuming proper choice of RSA parameters, $E$ and $D$ are inverse permutations of some set. Using that sole fact, we can prove that applying $i$ times $D$ and $j$ times $E$ to any elements of the set, in any order, is equivalent to applying $i-j$ times $D$ if $i-j$ is positive, or $j-i$ times $E$ otherwise. Proof is by induction on $i+j$.

Fact 3 in the question (which is true for any permutation on a finite set) was not used.

As rightly pointed in comment: RSA as actually used for encryption and decryption adds extra operations like padding to textbook RSA, and that destroys the property discussed; in particular, because the encrypted message's space is larger than the message's space.

• Given that the OP says that they're unfamiliar with RSA, it's probably worth noting explicitly that textbook RSA is not a secure cryptosystem on its own, and that adding in the extra pieces like padding that make it secure also destroys this property. – Ilmari Karonen Sep 24 '16 at 16:58
• Ah, was guessing the "textbook" part was about padding (comparing with matrix op.). Now I feel confident :) – packet0 Sep 25 '16 at 3:18

Considering the third bullet point, from a purely mathematical point of view you you are half-right:

• Since $\mathbb{Z}_N$ is not a field, $\mathbb{Z}_N \setminus \{0\}$ is not a cyclic group. It isn't even a group, because it has zero divisors. It is also not a cyclic permutation, because it has multiple nontrivial cyclic subgroups. The multiplicative group is actually a direct product of cyclic groups: $\mathbb{Z}_N^* \cong \mathbb{Z}_p^* \times \mathbb{Z}_q^*$. For more information see Wiki
• Applying one operation multiple times will eventually lead back to the original value, yes. Applying the exponentiation twice is $(m^{e})^e \mod N$, which is equivalent to $m^{ee} = m^{e^2} \mod N$, three times equivalent to $m^{e^3}\mod N$ and so on. In oder to find out how often we have to do that, we need to find $x$, s.t. $m^{e^x} = m^1$. That is equivalent to $e^x = 1 \mod \lambda(N)$. Note that $e$ is coprime to $\lambda(N)$, so $x$ does exist.

However, if you were able to find out this $x$, you could calculate a multiple of $\lambda(N)$, and then it's not too difficult to fatorize $N$. So for practical values, this value should be really hard to find (practically impossible).

On the note that you said RSA is new to you: The length of numbers used in RSA is usually large enough, that any kind of full search has no practical relevance. The success chance is much, much lower than $1$ divided by the number of atoms in the universe.

• The system is indeed cyclic: Encryption (plain RSA, no padding) defines a permutation on the numbers modulo $N$, and the system is the cyclic subgroup (of the symmetric group on the numbers modulo $N$) generated by this permutation (which equals the cyclic subgroup generated by decryption). – Hope that's a start Sep 26 '16 at 15:33
• @Hopethat'sastart There is no "cyclic system". The definition of a cyclic subgroup states that there is an element, which generates the whole group. That's not the case for RSA. A cyclic permutation allows only one non-trivial cycle (fixed points). And that isn't the case for RSA either. If the permutation consists of multiple cycles, that is not "cyclic". That is called a "permutation" in general. – tylo Sep 26 '16 at 18:06
• @Hopethat'sastart If you consider just that specific subgroup, which is generated by the permutation (Enc) and map composition, you have a cyclic subgroup. But that is quite trivial. And the symmetric group of permutations isn't cyclic in general. – tylo Sep 26 '16 at 18:18
• I guess it just depends on what one defines to be "the system". The specific permutation given by encryption/decryption and what it generates (my view and maybe OP's view; cyclic) or the set of all permutations that are given by exponentiations (your view; not cyclic). Both are legitimate views in my opinion. I just wanted to point out that 3) is OK seen from another point of view. – Hope that's a start Sep 27 '16 at 7:53