# Openssl library problem

I'm using openssl library and I've a problem with RSA_private_encrypt() and RSA_public_decrypt(). I'm using a key size of 256 bit with no padding of the input message. I've noted that when my input message ( random generated ) has 248 bit or less the functions fail with the following error :

Error code : error:04066084:lib(4):func(102):reason(132)

The reason code doesn't mean a lot for me :

#define RSA_R_DATA_TOO_LARGE_FOR_MODULUS                 132


Not working example :

Message :               00D6335705678D24251B26B1F1F60F8F864A5976806F0158589339B2920BE67F (         0)
Public exponent :       0000000000000000000000000000000000000000000000000000000000000003 (         0)
Private exponent :      8118EC84DB54C00F829C8C2D7E9A4278774582B5988C5BB0361521EF10BE4F33 (         0)
Output public key :     0000000000000000000000000000000000000000000000000000000000000000 (0000000000)
Output private key :    0000000000000000000000000000000000000000000000000000000000000000 (0000000000)


Working example :

Message :               01D6335705678D24251B26B1F1F60F8F864A5976806F0158589339B2920BE67F (         0)
Public exponent :       0000000000000000000000000000000000000000000000000000000000000003 (         0)
Private exponent :      8118EC84DB54C00F829C8C2D7E9A4278774582B5988C5BB0361521EF10BE4F33 (         0)
Output public key :     2E5FC4949004D0B9C976292329C767FA08567C18E4EB9A640EA8DE93F3739085 (         0)
Output private key :    C040AE99134F5C087C725CD5CE617F441101CB179F1EE6CAED2F8DFABC392EDB (         0)


In the 2 examples the public and private keys are the the same and in the message only one bit changes. Does anyone know if it is a bug/security precaution/something else ?

First, RSA "encrypt with private key" and "decrypt with public key" are semantically wrong; these don't provide confidentiality which is the purpose of encryption, but they can provide integrity which is the purpose of signature. The fact that RSA sign/verify operations are mathematically similar to encrypt/decrypt led to this misuse of terminology back in the 1980s, which unfortunately was (and remains) enshrined in the bottom-level function names in OpenSSL. (Higher level functions especially EVP fix this.)

Second, RSA without padding, so called "textbook" RSA, is not considered secure; it was also discovered back in the 1980s that it could be broken in cases that could not be reliably avoided. Actual use of RSA today uses padding, which differs between signing and encryption even though the math operation at the core (modular exponentiation) is almost the same. This means signing a value that is entirely random (unpadded) is basically useless.

Third, RSA 256 bits is much too small to be secure. I haven't worked out the numbers, but it could be factored very quickly by someone who wanted to break your key -- if you hadn't already published it. The recent FREAK attack (on SSL/TLS) demo factored 512 bits using AIR something like 1000 Amazon cloud machines in about 8 hours.

But those said, what you describe does work for me. I'm not sure what your "Output public key" is, but I can sign and verify/recover your second value (beginning 01) with the same signature you show for "Output private key", and I can sign and recover the value beginning with 00 the same way.

c:\temp>type fc30.c
#include <stdio.h>
#include <string.h>
#include "openssl/bn.h"
#include "openssl/rsa.h"

int digit (char c){ return c>='0'&&c<='9'? c-'0': (c&~0x20)-('A'-10); }
int hex2bin (unsigned char *x, const char *s)
{ int i = 0; while (s[0]&&s[1]) x[i++] = digit(s[0])<<4|digit(s[1]), s+=2; return i; }
void printbin (unsigned char *x, int n){ while(n--) printf("%02x",*x++); }

int main (void)
{
BIGNUM* n = BN_new();
BIGNUM* d = BN_new();
unsigned char xx[99], yy[99], zz[99];
RSA* rsa = RSA_new();
int t;

/* this doesn't track memory ownership as it should but enough for test */
BN_hex2bn (&d,"8118EC84DB54C00F829C8C2D7E9A4278774582B5988C5BB0361521EF10BE4F33");
rsa->n = n; rsa->d = d;
rsa->e = BN_new(); BN_set_word(rsa->e, 3);

t = hex2bin (xx,"00D6335705678D24251B26B1F1F60F8F864A5976806F0158589339B2920BE67F");
printf ("%d\n", t);
t = RSA_private_encrypt (t,xx, yy, rsa, RSA_NO_PADDING); /* REALLY SIGN */
printf ("%d:",t); printbin (yy,t); putchar('\n');
t = RSA_public_decrypt (t,yy, zz, rsa, RSA_NO_PADDING);
printf ("%d:",t); printbin (zz,t); putchar('\n');

xx[0]=1;
t = RSA_private_encrypt (t,xx, yy, rsa, RSA_NO_PADDING); /* DITTO */
printf ("%d:",t); printbin (yy,t); putchar('\n');
t = RSA_public_decrypt (t,yy, zz, rsa, RSA_NO_PADDING);
printf ("%d:",t); printbin (zz,t); putchar('\n');

return 0;
}

c:\temp>fc30
32
32:529cb2fc84470231a73026ab5d7f551965d8599f43492f992d87f429a8aaf311
32:00d6335705678d24251b26b1f1f60f8f864a5976806f0158589339b2920be67f
32:c040ae99134f5c087c725cd5ce617f441101cb179f1ee6caed2f8dfabc392edb
32:01d6335705678d24251b26b1f1f60f8f864a5976806f0158589339b2920be67f