When DIGIT_BIT is less than SIZEOF_LONG * CHAR_BIT, ASN1_INTEGER_get() can
return invalid value. For example, with trailing program, ASN1_INTEGER_get()
unexpectedly returns -268435449 (0xf0000007) on i386.
On the i386 platform (DIGIT_BIT=28), the input value 0x7fffffff is separated
into 0xfffffff and 0x7 and stored in the dp array of mp_int. Previously,
wolfSSL_BN_get_word_1() returned 0xfffffff shifted by 28 bits plus 0x7, so this
patch fixed it to return 0xfffffff plus 0x7 shifted by 28 bits.
int main(void)
{
ASN1_INTEGER *a;
long val;
int ret;
a = ASN1_INTEGER_new();
val = 0x7fffffff;
ret = ASN1_INTEGER_set(a, val);
if (ret != 1) {
printf("ret=%d\n", ret);
}
if (ASN1_INTEGER_get(a) != val) {
printf("ASN1_INTEGER_get=%ld\n", ASN1_INTEGER_get(a));
}
ASN1_INTEGER_free(a);
return 0;
}
Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
./configure --with-liboqs --enable-all --disable-psk --enable-intelasm --enable-aesni --enable-sp-math-all --enable-sp-asm CFLAGS="-O3"
Yeilds the following erorr:
src/internal.c: In function ‘DoServerKeyExchange’:
src/internal.c:24487:28: error: ‘sigAlgo’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
24487 | if (sigAlgo == ed448_sa_algo &&
| ^
This fixes it.
