* Adds ./configure "--with-intelqa=../QAT1.6”, port files, memory management and README.md (see wolfcrypt/src/port/intel/). * Added Intel QAT support for RSA public/private (CRT/non-CRT), AES CBC/GCM, ECDH/ECDSA, DH, DES3, SHA, SHA224, SHA256, SHA384, SHA512, MD5 and HMAC. * wolfSSL async enabled all client and server: PKI, Encrypt/Decrypt, Hashing/HMAC and Certificate Sign/Verify. * wolfSSL async support in functions: Encrypt, Decrypt, VerifyMAC, BuildMessage, ConfirmSignature, DoCertificate, ParseCertRelative, and MakeSignature. * wolfCrypt test and benchmark async support added for all HW acceleration. * wolfCrypt benchmark multi-threading support. * Added QuickAssist memory overrides for XMALLOC, XFREE and XREALLOC. XREALLOC determines if existing pointer needs reallocated for NUMA. * Refactor to make sure “heap” is available for async dev init. * Added async support for all examples for connect, accept, read and write. * Added new WC_BIGINT (in wolfmath.c) for async hardware support. * Added async simulator tests for DES3 CBC, AES CBC/GCM. * Added QAT standalone build for unit testing. * Added int return code to SHA and MD5 functions. * Refactor of the async stack variable handling, so async operations have generic args buffer area and cleanup function pointer. * Combined duplicate code for async push/pop handling. * Refactor internal.c to add AllocKey / FreeKey. * Refactor of hash init/free in TLS to use InitHashes and FreeHashes. * Refactor of the async event->context to use WOLF_EVENT_TYPE_ASYNC_WOLFSSL for WOLFSSL* and WOLF_EVENT_TYPE_ASYNC_WOLFCRYPT for WC_ASYNC_DEV*. * Suppress error message for WC_PENDING_E. * Implemented "wolfSSL_EVP_MD_CTX_init" to do memset. * Cleanup of the openssl compat CTX sizes when async is enabled. * Cleanup of AES, DES3, DH, SHA, MD5, DES3, DH, HMAC, MD5 for consistency and readability. * Cleanup of the OPAQUE_LEN. * Cleanup to use ENCRYPT_LEN instead of sizeof(ssl->arrays.preMasterSecret). * Changed ssl->arrays.preMasterSecret to use XMALLOC (accelerates HW operations) * Reduce verbosity with debug enabled for "GetMyVersion", "wolfSSL Using RSA OAEP padding" and "wolfSSL Using RSA PKCSV15 padding". * Updated RSA un-padding error message so its different than one above it for better debugging. * Added QAT async enables for each algorithm. * Refactor of the async init to use _ex. * Added WC_ASYNC_THRESH_NONE to allow bypass of the async thresholds for testing. * Reformatted the benchmark results: PKI: "RSA 2048 private HW 18522 ops took 1.003 sec, avg 0.054 ms, 18467.763 ops/sec" Crypto/Hashing: SHA-256 SW 350 megs took 1.009 seconds, 346.946 MB/s Cycles per byte = 9.87 * Added min execution time for all benchmarks. * Moved wc_*GetHash and wc_*RestorePos to appropriate files so use of isCopy flag is local. * Fix for ECC sign status sometimes being invalid due to uninitialized ECC digest in benchmark. * Added new DECLARE_VAR/FREE_VAR and DECLARE_ARRAY/FREE_ARRAY macros for helping setup test/benchmark variables to accelerate async. * Added NO_SW_BENCH option to only run HW bench. * Added support for PRNG to use hardware SHA256 if _wc devId provided. * Fix to prevent curve tests from running against wrong curve sizes. Changed wc_ecc_set_curve to match on exact size. * Added the wc_*GetHash calls to the wolfCrypt tests. * Added async hardware start/stop to wolfSSL init/cleanup. * Refactor to add wc_*Copy for hashing context (for async), which replaces wc_*RestorePos. * Fixes for building with TI hashing (including: SHA224, missing new API’s and building with dummy build for non hw testing). Note: We need to add build test for this `./configure CFLAGS="-DWOLFSSL_TI_HASH -DTI_DUMMY_BUILD”`. * Added arg checks on wc_*GetHash and wc_*Copy. * Cleanup of the BuildMD5, BuildSHA, BuildMD5_CertVerify and BuildSHA_CertVerify functions. * Added new ./configure --enable-asyncthreads, to allow enable/disable of the async threading support. If --enable-asynccrypt set this will be enabled by default if pthread is supported. Allows multi-threaded benchmarks with async simulator. * Added checks for all hashing to verify valid ->buffLen. * Fix for SHA512 scan-build warning about un-initialized “W_X”. * Fix for valgrind un-initialized use of buffer in AllocDer (der->buffer) and BuildTlsFinished handshake_hash. * Refactor of the benchmarking to use common function for start, check and finish of the stats. * Fixed issue with ECC cache loading in multi-threading. * Fix bug with AESNI not aligned code that assumes XMALLOC is 16-byte aligned. * Added new WC_ASYNC_NO_… options to allow disabling of individual async algorithms. New defines are: WC_ASYNC_NO_CRYPT, WC_ASYNC_NO_PKI and WC_ASYNC_NO_HASH. Additionally each algorithm has a WC_ASYNC_NO_[ALGO] define. * Added “wolfSSL_GetAllocators” API and fixed the wolfCrypt memcb_test so it restores callback pointers after test is complete (fixes issue with using custom allocators and test breaking it).
286 lines
6.3 KiB
C
286 lines
6.3 KiB
C
/* wolfevent.c
|
|
*
|
|
* Copyright (C) 2006-2016 wolfSSL Inc.
|
|
*
|
|
* This file is part of wolfSSL.
|
|
*
|
|
* wolfSSL is free software; you can redistribute it and/or modify
|
|
* it under the terms of the GNU General Public License as published by
|
|
* the Free Software Foundation; either version 2 of the License, or
|
|
* (at your option) any later version.
|
|
*
|
|
* wolfSSL is distributed in the hope that it will be useful,
|
|
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
* GNU General Public License for more details.
|
|
*
|
|
* You should have received a copy of the GNU General Public License
|
|
* along with this program; if not, write to the Free Software
|
|
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1335, USA
|
|
*/
|
|
|
|
#ifdef HAVE_CONFIG_H
|
|
#include <config.h>
|
|
#endif
|
|
|
|
#include <wolfssl/wolfcrypt/settings.h>
|
|
|
|
|
|
#ifdef HAVE_WOLF_EVENT
|
|
|
|
#include <wolfssl/internal.h>
|
|
#include <wolfssl/error-ssl.h>
|
|
#include <wolfssl/wolfcrypt/error-crypt.h>
|
|
|
|
#include <wolfssl/wolfcrypt/wolfevent.h>
|
|
|
|
|
|
int wolfEvent_Init(WOLF_EVENT* event, WOLF_EVENT_TYPE type, void* context)
|
|
{
|
|
if (event == NULL) {
|
|
return BAD_FUNC_ARG;
|
|
}
|
|
|
|
if (event->pending) {
|
|
WOLFSSL_MSG("event already pending!");
|
|
return BAD_COND_E;
|
|
}
|
|
|
|
XMEMSET(event, 0, sizeof(WOLF_EVENT));
|
|
event->type = type;
|
|
event->context = context;
|
|
|
|
return 0;
|
|
}
|
|
|
|
int wolfEvent_Poll(WOLF_EVENT* event, WOLF_EVENT_FLAG flags)
|
|
{
|
|
int ret = BAD_COND_E;
|
|
|
|
/* Check hardware */
|
|
#ifdef WOLFSSL_ASYNC_CRYPT
|
|
if (event->type >= WOLF_EVENT_TYPE_ASYNC_FIRST &&
|
|
event->type <= WOLF_EVENT_TYPE_ASYNC_LAST)
|
|
{
|
|
ret = wolfAsync_EventPoll(event, flags);
|
|
}
|
|
#endif /* WOLFSSL_ASYNC_CRYPT */
|
|
|
|
return ret;
|
|
}
|
|
|
|
int wolfEventQueue_Init(WOLF_EVENT_QUEUE* queue)
|
|
{
|
|
int ret = 0;
|
|
|
|
if (queue == NULL) {
|
|
return BAD_FUNC_ARG;
|
|
}
|
|
|
|
XMEMSET(queue, 0, sizeof(WOLF_EVENT_QUEUE));
|
|
#ifndef SINGLE_THREADED
|
|
ret = wc_InitMutex(&queue->lock);
|
|
#endif
|
|
return ret;
|
|
}
|
|
|
|
|
|
int wolfEventQueue_Push(WOLF_EVENT_QUEUE* queue, WOLF_EVENT* event)
|
|
{
|
|
int ret;
|
|
|
|
if (queue == NULL || event == NULL) {
|
|
return BAD_FUNC_ARG;
|
|
}
|
|
|
|
#ifndef SINGLE_THREADED
|
|
if ((ret = wc_LockMutex(&queue->lock)) != 0) {
|
|
return ret;
|
|
}
|
|
#endif
|
|
|
|
/* Setup event */
|
|
event->next = NULL;
|
|
event->pending = 1;
|
|
|
|
ret = wolfEventQueue_Add(queue, event);
|
|
|
|
#ifndef SINGLE_THREADED
|
|
wc_UnLockMutex(&queue->lock);
|
|
#endif
|
|
|
|
return ret;
|
|
}
|
|
|
|
int wolfEventQueue_Pop(WOLF_EVENT_QUEUE* queue, WOLF_EVENT** event)
|
|
{
|
|
int ret = 0;
|
|
|
|
if (queue == NULL || event == NULL) {
|
|
return BAD_FUNC_ARG;
|
|
}
|
|
|
|
#ifndef SINGLE_THREADED
|
|
/* In single threaded mode "event_queue.lock" doesn't exist */
|
|
if ((ret = wc_LockMutex(&queue->lock)) != 0) {
|
|
return ret;
|
|
}
|
|
#endif
|
|
|
|
/* Pop first item off queue */
|
|
*event = queue->head;
|
|
ret = wolfEventQueue_Remove(queue, *event);
|
|
|
|
#ifndef SINGLE_THREADED
|
|
wc_UnLockMutex(&queue->lock);
|
|
#endif
|
|
|
|
return ret;
|
|
}
|
|
|
|
/* assumes queue is locked by caller */
|
|
int wolfEventQueue_Add(WOLF_EVENT_QUEUE* queue, WOLF_EVENT* event)
|
|
{
|
|
if (queue == NULL || event == NULL) {
|
|
return BAD_FUNC_ARG;
|
|
}
|
|
|
|
if (queue->tail == NULL) {
|
|
queue->head = event;
|
|
}
|
|
else {
|
|
queue->tail->next = event;
|
|
event->prev = queue->tail;
|
|
}
|
|
queue->tail = event; /* add to the end either way */
|
|
queue->count++;
|
|
|
|
return 0;
|
|
}
|
|
|
|
/* assumes queue is locked by caller */
|
|
int wolfEventQueue_Remove(WOLF_EVENT_QUEUE* queue, WOLF_EVENT* event)
|
|
{
|
|
int ret = 0;
|
|
|
|
if (queue == NULL || event == NULL) {
|
|
return BAD_FUNC_ARG;
|
|
}
|
|
|
|
if (event == queue->head && event == queue->tail) {
|
|
queue->head = NULL;
|
|
queue->tail = NULL;
|
|
}
|
|
else if (event == queue->head) {
|
|
queue->head = event->next;
|
|
queue->head->prev = NULL;
|
|
}
|
|
else if (event == queue->tail) {
|
|
queue->tail = event->prev;
|
|
queue->tail->next = NULL;
|
|
}
|
|
else {
|
|
WOLF_EVENT* next = event->next;
|
|
WOLF_EVENT* prev = event->prev;
|
|
next->prev = prev;
|
|
prev->next = next;
|
|
}
|
|
queue->count--;
|
|
|
|
return ret;
|
|
}
|
|
|
|
int wolfEventQueue_Poll(WOLF_EVENT_QUEUE* queue, void* context_filter,
|
|
WOLF_EVENT** events, int maxEvents, WOLF_EVENT_FLAG flags, int* eventCount)
|
|
{
|
|
WOLF_EVENT* event;
|
|
int ret = 0, count = 0;
|
|
|
|
if (queue == NULL) {
|
|
return BAD_FUNC_ARG;
|
|
}
|
|
|
|
#ifndef SINGLE_THREADED
|
|
/* In single threaded mode "event_queue.lock" doesn't exist */
|
|
if ((ret = wc_LockMutex(&queue->lock)) != 0) {
|
|
return ret;
|
|
}
|
|
#endif
|
|
|
|
/* itterate event queue */
|
|
for (event = queue->head; event != NULL; event = event->next)
|
|
{
|
|
/* optional filter based on context */
|
|
if (context_filter == NULL || event->context == context_filter) {
|
|
|
|
/* poll event */
|
|
ret = wolfEvent_Poll(event, flags);
|
|
if (ret < 0) break; /* exit for */
|
|
|
|
/* If event is done then process */
|
|
if (event->done) {
|
|
/* remove from queue */
|
|
ret = wolfEventQueue_Remove(queue, event);
|
|
if (ret < 0) break; /* exit for */
|
|
|
|
/* return pointer in 'events' arg */
|
|
if (events) {
|
|
events[count] = event; /* return pointer */
|
|
}
|
|
count++;
|
|
|
|
/* check to make sure our event list isn't full */
|
|
if (events && count >= maxEvents) {
|
|
break; /* exit for */
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
#ifndef SINGLE_THREADED
|
|
wc_UnLockMutex(&queue->lock);
|
|
#endif
|
|
|
|
/* return number of properly populated events */
|
|
if (eventCount) {
|
|
*eventCount = count;
|
|
}
|
|
|
|
return ret;
|
|
}
|
|
|
|
int wolfEventQueue_Count(WOLF_EVENT_QUEUE* queue)
|
|
{
|
|
int ret;
|
|
|
|
if (queue == NULL) {
|
|
return BAD_FUNC_ARG;
|
|
}
|
|
|
|
#ifndef SINGLE_THREADED
|
|
/* In single threaded mode "event_queue.lock" doesn't exist */
|
|
if ((ret = wc_LockMutex(&queue->lock)) != 0) {
|
|
return ret;
|
|
}
|
|
#endif
|
|
|
|
ret = queue->count;
|
|
|
|
#ifndef SINGLE_THREADED
|
|
wc_UnLockMutex(&queue->lock);
|
|
#endif
|
|
|
|
return ret;
|
|
}
|
|
|
|
void wolfEventQueue_Free(WOLF_EVENT_QUEUE* queue)
|
|
{
|
|
if (queue) {
|
|
#ifndef SINGLE_THREADED
|
|
wc_FreeMutex(&queue->lock);
|
|
#endif
|
|
}
|
|
}
|
|
|
|
#endif /* HAVE_WOLF_EVENT */
|