deal with SSL_ERROR_WANT_ in client connect action

"4b0e01f Retry SSL_connect when SSL_get_error requests it. " from David Galeano
noticed the problem that client connect may receive SSL_ERROR_WANT_* from
SSL_connect, which is basically WOULDBLOCK.  That patch tried to deal with it
by blocking in a while(1) until the condition went away.

That's problematic because of it blocks service of anything else (including
the host application sockets in the external socket poll sharing case) for
up to 5s controlled by conditions at one client.

After fiddling with and researching this, the actual problem with the code is
we are not getting the SSL layer error correctly, it is not contained in the
code returned from the Connect api directly.

I was unable to get a renegotiation forced on my modern SSL libs, it complained
about protocol error are reopened the connection instead.  So I think the stuff
found in the docs and the web about the SSL_ERROR_WANT_ is probably not something
we will see in reality (if we check the right error code...)

Signed-off-by: Andy Green <andy.green@linaro.org>

lib/client-handshake.c

@@ -123,9 +123,15 @@ struct libwebsocket *__libwebsocket_client_connect_2(
 	 * provoke service to issue the handshake directly
 	 * we need to do it this way because in the proxy case, this is the
 	 * next state and executed only if and when we get a good proxy
-	 * response inside the state machine
+	 * response inside the state machine... but notice in SSL case this
+	 * may not have sent anything yet with 0 return, and won't until some
+	 * many retries from main loop.  To stop that becoming endless,
+	 * cover with a timeout.
 	 */
 
+	libwebsocket_set_timeout(wsi,
+		PENDING_TIMEOUT_SENT_CLIENT_HANDSHAKE, AWAITING_TIMEOUT);
+
 	wsi->mode = LWS_CONNMODE_WS_CLIENT_ISSUE_HANDSHAKE;
 	pfd.fd = wsi->sock;
 	pfd.revents = POLLIN;

lib/libwebsockets.c

@@ -1834,7 +1834,27 @@ bail_prox_listener:
 
 	case LWS_CONNMODE_WS_CLIENT_ISSUE_HANDSHAKE:
 
+		/*
+		 * we are under PENDING_TIMEOUT_SENT_CLIENT_HANDSHAKE
+		 * timeout protection set in client-handshake.c
+		 */
+
 	#ifdef LWS_OPENSSL_SUPPORT
+
+		/*
+		 * take care of our libwebsocket_callback_on_writable
+		 * happening at a time when there's no real connection yet
+		 */
+
+		pollfd->events &= ~POLLOUT;
+
+		/* external POLL support via protocol 0 */
+		context->protocols[0].callback(context, wsi,
+			LWS_CALLBACK_CLEAR_MODE_POLL_FD,
+			(void *)(long)wsi->sock, NULL, POLLOUT);
+
+		/* we can retry this... so just cook the SSL BIO the first time */
+
 		if (wsi->use_ssl && !wsi->ssl) {
 
 			wsi->ssl = SSL_new(context->ssl_client_ctx);
@@ -1848,8 +1868,33 @@ bail_prox_listener:
 		}		
 
 		if (wsi->use_ssl) {
-			if (SSL_connect(wsi->ssl) <= 0) {
+			n = SSL_connect(wsi->ssl);
 
+			if (n < 0) {
+				n = SSL_get_error(wsi->ssl, n);
+
+				if (n == SSL_ERROR_WANT_READ ||
+					n == SSL_ERROR_WANT_WRITE) {
+					/*
+					 * wants us to retry connect due to state of the
+					 * underlying ssl layer... but since it may be
+					 * stalled on blocked write, no incoming data may
+					 * arrive to trigger the retry.  Force (possibly
+					 * many if the SSL state persists in returning the
+					 * condition code, but other sockets are getting
+					 * serviced inbetweentimes) us to get called back
+					 * when writable.
+					 */
+
+					lwsl_info("SSL_connect -> SSL_ERROR_WANT_... retrying\n");
+					libwebsocket_callback_on_writable(context, wsi);
+
+					return 0; /* no error */
+				}
+				n = -1;
+			}
+
+			if (n <= 0) {
 				/*
 				 * retry if new data comes until we
 				 * run into the connection timeout or win

lib/private-libwebsockets.h

@@ -305,6 +305,7 @@ enum pending_timeout {
 	PENDING_TIMEOUT_AWAITING_PING,
 	PENDING_TIMEOUT_CLOSE_ACK,
 	PENDING_TIMEOUT_AWAITING_EXTENSION_CONNECT_RESPONSE,
+	PENDING_TIMEOUT_SENT_CLIENT_HANDSHAKE,
 };