tl;dr: I discuss the issues related to small exponents, how to improve the success rate of an attack on RSA, how the square root relates to the hardness of the RSA problem, and its connection with Shor’s algorithm. Many of the materials discussed below are thanks to the lecture notes and coursework that I have taken in Introduction to Cryptography.

Prerequisites

Small Exponent Attack

Alice wants to send a (k − 1)-bit message, mm, to Bob, Cathy, and Dave. Bob, Cathy, and Dave have distinct k-bit RSA keys, NBN_B, NCN_C, and NDN_D, and they all use the same exponent, e=3e = 3. Alice uses plain textbook RSA to encrypt mm under each key, resulting in three ciphertexts cBc_B, cCc_C, and cDc_D, which she sends to them. Eve intercepts all three ciphertexts. Show that Eve can recover the plaintext mm.


There are two cases to consider:

  1. NAN_A, NBN_B, and NCN_C are not pairwise coprime.

    Without loss of generality, suppose NAN_A and NBN_B are not coprime. Then one can use gcd(NA,NB)\gcd(N_A, N_B) to factor both NAN_A and NBN_B. Once factored, computing ϕ(NA)\phi(N_A) is straightforward, which then enables decryption of the ciphertext memodNAm^e \bmod N_A.

  2. NAN_A, NBN_B, and NCN_C are pairwise coprime.

    In this case, we have:

    cA=m3modNAcB=m3modNBcC=m3modNC\begin{align*}c_A &= m^3 \bmod N_A \\c_B &= m^3 \bmod N_B \\c_C &= m^3 \bmod N_C \\\end{align*}

    Using the Chinese Remainder Theorem (CRT), we can compute a value cc' such that

    c=m3mod(NANBNC)c' = m^3 \bmod (N_A N_B N_C)

    Since mm is smaller than any of the moduli, m3<NANBNCm^3 < N_A N_B N_C. Therefore, the cube root of cc' can be taken directly to recover mm.

Improve RSA Attack Success Rate

Let (N,e)(N, e) be an RSA key, where 2<e<N2 < e < N. Suppose A\mathcal{A} is a probabilistic algorithm that runs in time tt with

PrrZN[A(N,e,remodN)=r]=1%.\Pr_{r \leftarrow \mathbb{Z}_N} \left[ \mathcal{A}(N, e, r^e \bmod N) = r \right] = 1\%.

Show that A\mathcal{A} can be used as a subroutine to construct an algorithm B\mathcal{B} such that

PrrZN[B(N,e,remodN)=r]=99%.\Pr_{r \leftarrow \mathbb{Z}_N} \left[ \mathcal{B}(N, e, r^e \bmod N) = r \right] = 99\%.

To build such an algorithm B\mathcal{B}, we rely on the fact that RSA is multiplicatively homomorphic. With this in mind, one can devise an algorithm B(N,e,y)\mathcal{B}(N, e, y) as follows:

  • Repeat:
    • Sample s$ZNs \xleftarrow{$} \mathbb{Z}_N.
    • Compute xA(N,e,ysemodN)x \leftarrow \mathcal{A}(N, e, ys^e \bmod N).
  • Until xe=ysex^e = ys^e.

Then, by using the homomorphic property, recover rr as:

rxs1modN.r \leftarrow x \cdot s^{-1} \bmod N.

By repeating the algorithm A\mathcal{A} a bounded number of times, we can design B\mathcal{B} with a success probability of 99%.

Square Root and RSA

Let’s begin by examining an interesting attack.

RSA Key Leakage Vulnerability

Suppose Bob uses the RSA cryptosystem and has a key pair with public key ee and private key dd. If Bob accidentally leaks his private key and decides to generate a new public/private key pair without generating a new modulus, show that this approach is insecure.


To prove that this approach is insecure, we will show that, given dd, one can factor NN (assuming NN is the product of two odd primes).

  1. Given dd, compute k=ed1k = ed - 1 using the public key ee.
  2. By definition, kk is divisible by ϕ(N)\phi(N). Since Euler’s totient function ϕ(N)\phi(N) is even, kk is even, and hence k2\frac{k}{2} is an integer.
  3. By Euler’s theorem, for every gZNg \in \mathbb{Z}_N^*, we have gk=1g^k = 1. Therefore, gk2g^{\frac{k}{2}} is a square root of 1modN1 \bmod N.

Properties of Square Roots

Before proceeding with the proof, let’s study the properties of the square roots of 11, which we denote by xx.

Using the CRT, since N=pqN = pq where pp and qq are primes, solving

x21(modN)x^2 \equiv 1 \pmod{N}

is equivalent to solving

x21(modp)andx21(modq).x^2 \equiv 1 \pmod{p} \quad \text{and} \quad x^2 \equiv 1 \pmod{q}.

For a prime pp, the solutions to x21(modp)x^2 \equiv 1 \pmod{p} are x±1(modp)x \equiv \pm 1 \pmod{p}. Similarly for qq. Hence, there are four solutions modulo NN, determined by the combinations of ±1(modp)\pm 1 \pmod{p} and ±1(modq)\pm 1 \pmod{q}:

  1. n11(modp)n_1 \equiv 1 \pmod{p} and n11(modq)n_1 \equiv 1 \pmod{q}, i.e., n11(modN)n_1 \equiv 1 \pmod{N}.
  2. n21(modp)n_2 \equiv -1 \pmod{p} and n21(modq)n_2 \equiv -1 \pmod{q}, i.e., n21(modN)n_2 \equiv -1 \pmod{N}.
  3. n31(modp)n_3 \equiv 1 \pmod{p} and n31(modq)n_3 \equiv -1 \pmod{q}.
  4. n41(modp)n_4 \equiv -1 \pmod{p} and n41(modq)n_4 \equiv 1 \pmod{q}.

Each nin_i satisfies ni21(modN)n_i^2 \equiv 1 \pmod{N} because ni21n_i^2 \equiv 1 modulo both pp and qq.

The nontrivial solutions n3n_3 and n4n_4 are especially important for factoring NN. Consider n3n_3:

  • Since n31(modp)n_3 \equiv 1 \pmod{p}, we have n310(modp)n_3 - 1 \equiv 0 \pmod{p}; hence, pp divides n31n_3 - 1.
  • Since n31(modq)n_3 \equiv -1 \pmod{q}, we have n312(modq)n_3 - 1 \equiv -2 \pmod{q}, and since qq is odd, 2≢0(modq)-2 \not\equiv 0 \pmod{q}. Thus, qq does not divide n31n_3 - 1.

Therefore, gcd(n31,N)=p\gcd(n_3 - 1, N) = p. A similar argument applies to n4n_4, except it yields qq.

However, finding a nontrivial square root of 1 is not straightforward in RSA. In fact, it can be shown that for every gZNg \in \mathbb{Z}_N^*, gk2=1modNg^{\frac{k}{2}} = 1 \bmod N. To explain this, express p1=2tprpp-1 = 2^{\tp} r_p and q1=2tqrqq-1 = 2^{\tq} r_q with tp,tq1\tp, \tq \geq 1. Then,

k=α2tp+tqrprq,k = \alpha \cdot 2^{\tp + \tq} r_p r_q,

where α\alpha is a positive integer and tp+tq2\tp + \tq \geq 2. This implies that p1p - 1 divides k2\frac{k}{2} and similarly for q1q - 1. By the CRT and Euler’s Theorem, it follows that

gZN,  gk21modN.\forall g \in \mathbb{Z}_N^*, \; g^{\frac{k}{2}} \equiv 1 \bmod N.

On the bright side, if k2\frac{k}{2} is even, we can iterate and consider gk4g^{\frac{k}{4}}, gk8g^{\frac{k}{8}}, and so on to search for a nontrivial square root of 1. Formally, writing k=2trk = 2^t r where ttp+tqt \geq \tp + \tq and rr is an odd integer, we can examine the sequence:

gk2, gk4, , gk2t.g^{\frac{k}{2}},\ g^{\frac{k}{4}},\ \dots,\ g^{\frac{k}{2^t}}.

If, for any ii, the value gcd(gk2i1,N)\gcd\left(g^{\frac{k}{2^i}} - 1, N\right) is neither 1 nor NN, then we can successfully factor NN[1].

But what is the probability that such an event occurs when we randomly select gg?

We will now prove a lower bound: such an event occurs with probability at least 12\frac{1}{2}. Let tmax=max(tp,tq)t_{\text{max}} = \max(\tp, \tq).

A strawman example: First, consider k=2trk' = 2^{t'} r where ttmaxt' \geq t_{\text{max}}. Since ttmaxt' \geq t_{\text{max}}, both p1p - 1 and q1q - 1 divide kk'. Thus, for any lttmaxl \leq t - t_{\text{max}}, gk2lg^{\frac{k}{2^l}} will always be 1. This suggests that we should instead look at the case when t<tmaxt' < t_{\text{max}}.


Before proceeding further, we prove the following lemma:

Lemma

  1. The multiplicative group Zp\mathbb{Z}_p^* (with pp prime) is cyclic; that is, it has a generator.
  2. Zp\mathbb{Z}_p^ has a subgroup SS of order p12\frac{p-1}{2}, and for every eZpe \in \mathbb{Z}_p^, we have ep121modpe^{\frac{p-1}{2}} \equiv 1 \bmod p if and only if eSe \in S.

Proof.

  1. This follows from the standard result that the multiplicative group of a finite field is cyclic. Let gg denote a generator of this group.

  2. The subgroup SS exists and can be generated by g2g^2. Its order is p12\frac{p-1}{2} because p12\frac{p-1}{2} is the smallest integer satisfying (g2)p12=gp11modp\left(g^2\right)^{\frac{p-1}{2}} = g^{p-1} \equiv 1 \bmod p. The rest of the lemma follows:

    • (\Leftarrow) If eSe \in S, then e=(g2)ce = (g^2)^c for some integer cc, so

      ep12=(g2)cp12=1c=1modp.e^{\frac{p-1}{2}} = \left(g^2\right)^{c \frac{p-1}{2}} = 1^c = 1 \bmod p.
    • (\Rightarrow) Conversely, if ep121modpe^{\frac{p-1}{2}} \equiv 1 \bmod p and e=gce = g^c, then

      gcp121modp.g^{c \cdot \frac{p-1}{2}} \equiv 1 \bmod p.

      Since gg has order p1p-1, it must be that cp12c \cdot \frac{p-1}{2} is a multiple of p1p-1; hence, cc is even. Writing c=2kc = 2k, we have e=g2k=(g2)ke = g^{2k} = (g^2)^k, so eSe \in S. This also implies that if eSe \notin S, then ep121modpe^{\frac{p-1}{2}} \equiv -1 \bmod p.


Now, consider the case t=tmax1t' = t_{\text{max}} - 1. There are two cases to analyze:

  1. Case 1: tp<tq\tp < \tq (without loss of generality).

    Since tmax=tqt_{\text{max}} = \tq, we have tpt\tp \leq t' and thus p1p - 1 divides kk'. This implies gk1modpg^{k'} \equiv 1 \bmod p. To obtain a nontrivial square root, we require gk1modqg^{k'} \equiv -1 \bmod q. Writing k=q12rk' = \frac{q-1}{2}r' (with rr' odd), by the lemma, if gg is in the subgroup SS of Zq\mathbb{Z}_q^* (which has order q12\frac{q-1}{2}), then gk1modqg^{k'} \equiv 1 \bmod q; if gSg \notin S, then gk(1)r=1modqg^{k'} \equiv (-1)^{r'} = -1 \bmod q. Therefore, for a randomly sampled gg,

    Pr[gk1modq]=1Pr[gS]=1q12q1=12.\Pr[g^{k'} \equiv -1 \bmod q] = 1 - \Pr[g \in S] = 1 - \frac{\frac{q-1}{2}}{q-1} = \frac{1}{2}.
  2. Case 2: tp=tq\tp = \tq.

    In this case, to obtain a nontrivial square root of 1, either

    • gk1modpg^{k'} \equiv 1 \bmod p and gk1modqg^{k'} \equiv -1 \bmod q, or
    • gk1modpg^{k'} \equiv -1 \bmod p and gk1modqg^{k'} \equiv 1 \bmod q.

    By a similar analysis as in Case 1, the total probability of a nontrivial square root occurring is

    1212+1212=12.\frac{1}{2} \cdot \frac{1}{2} + \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{2}.

Thus, regardless of tp\tp and tq\tq, we have:

Pr[gcd(gk1,N) is a factor of N]=12.\Pr\left[\gcd\left(g^{k'} - 1, N\right) \text{ is a factor of } N\right] = \frac{1}{2}.

Therefore,

Pr[i=1lgcd(gk2i1,N) is a factor of N]12.\Pr\left[\bigcup_{i=1}^{l} \gcd\left(g^{\frac{k}{2^i}} - 1, N\right) \text{ is a factor of } N\right] \geq \frac{1}{2}.

This completes the proof.

You can also programmatically verify the above proof using this simulation. The simulation verifies the above proof and additionally provides another method to simulate the probability of factorizing NN by checking all the elements in the sequence.

Proof (Continued)

To factor NN, we can randomly sample group elements gg until we find one that yields a nontrivial square root of 1. Based on the analysis above, this happens with a high probability after only a few tries. Ultimately, the nontrivial square root is used to factor NN[2].

Shor’s Algorithm

The idea behind Shor’s algorithm (period finding) is also connected to using square roots to factor NN.

Suppose we use Shor’s algorithm to find an rr such that

f(x+r)=f(x),f(x + r) = f(x),

that is,

(gx)r=gxgr11modN.(g^x)^r = g^x \quad \Rightarrow \quad g^{r-1} \equiv 1 \bmod N.

Then, by the previous arguments, with probability at least 12\frac{1}{2}, we can use gr12, , gr12lg^{\frac{r-1}{2}},\ \dots,\ g^{\frac{r-1}{2^l}} to factor NN. Consequently, one can repeatedly sample elements and apply Shor’s algorithm until success is achieved.

Another perspective on how a nontrivial square root aids in factoring NN is to note that for an element with gr11modNg^{r-1} \equiv 1 \bmod N, we have:

(gr121)(gr12+1)0modN.\left(g^{\frac{r-1}{2}} - 1\right) \left(g^{\frac{r-1}{2}} + 1\right) \equiv 0 \bmod N.

Thus, one of the factors in the parenthesis will be a prime factor of NN (provided gr12±1g^{\frac{r-1}{2}} \neq \pm 1).

Next Steps

  1. The sequence computation and gcd calculation can be done in polynomial time. See Proof of Fact 1.

  2. This is not the only method to attack RSA when both ee and dd are known. This post details an alternative approach that estimates ed1ϕ(N)\frac{ed - 1}{\phi(N)} to compute ϕ(N)\phi(N), which then allows one to deduce p+q=Nϕ(N)+1p+q = N - \phi(N) + 1 and factor NN.