(Includes fixes from Yichen Gu)
Currently the incoming ebuf is always replaced to point to either a whole
buflist segment, or up to the (pt_serv_buf - LWS_PRE) length in the pt_serv_buf.
This is called on path for handling http read... some user code reasonably wants to
restrict the read size to what it can handle.
Change the other lws_buflist_aware_read() callers to zero ebuf before calling, and for
those have it keep the current behaviour; but if non-NULL ebuf.token on incoming, as
in http read path case, restrict both reported len of buflist content and the read length
to the incoming ebuf.len so the user code can control what it will get at one time.
Additionally muxed protocol wsi have no choice but to read what was sent to them
since it's HOL-blocking for other streams and its own WINDOW_UPDATEs. So add an
internal param to lws_buflist_aware_read() forcing read even if buflist content
is available.
This provides support to build lws using the linkit 7697 public SDK
from here https://docs.labs.mediatek.com/resource/mt7687-mt7697/en/downloads
This toolchain has some challenges, its int32_t / uint32_t are long,
so assumptions about format strings for those being %u / %d / %x all
break. This fixes all the cases for the features enabled by the
default cmake settings.
Freertos + lwip doesn't support pipe2() or pipe()... implement a "pipe"
based on two UDP sockets, one listening on 127.0.0.1:54321 and the other
doing a sendto() there of a single byte to interrupt the event loop wait.
Re-use the arrangements for actual pipe fds and pipe role to deliver
lws_cancel_service() functionality using this.
There are some minor public api type improvements rather than cast everywhere
inside lws and user code to work around them... these changed from int to
size_t
- lws_buflist_use_segment() return
- lws_tokenize_t .len and .token_len
- lws_tokenize_cstr() length
- lws_get_peer_simple() namelen
- lws_get_peer_simple_fd() namelen, int fd -> lws_sockfd_type fd
- lws_write_numeric_address() len
- lws_sa46_write_numeric_address() len
These changes are typically a NOP for user code
In the case code is composed into a single process, but it isn't monolithic in the
sense it's made up of modular "applications" that are written separate projects,
provide a way for the "applications" to request a callback from the lws event loop
thread context safely.
From the callback the applications can set up their operations on the lws event
loop and drop their own thread.
Since it requires system-specific locking to be threadsafe, provide a non-threadsafe
helper and then indirect the actual usage through a user-defined lws_system ops
function pointer that wraps the unsafe api with the system locking to make it safe.
Since version 3.1.0 and commit aa4143aebd,
-pthread is unconditionally added to CMAKE_REQUIRED_FLAGS even if
pthread.h is not found, this will result in a build failure with openssl
if the toolchain doesn't support threads:
[ 5%] Building C object CMakeFiles/websockets_shared.dir/lib/core/lws_dll2.c.o
In file included from /home/buildroot/autobuild/instance-2/output-1/build/libwebsockets-3.2.0/include/libwebsockets.h:570,
from /home/buildroot/autobuild/instance-2/output-1/build/libwebsockets-3.2.0/lib/core/private.h:130,
from /home/buildroot/autobuild/instance-2/output-1/build/libwebsockets-3.2.0/lib/core/lws_dll2.c:22:
/home/buildroot/autobuild/instance-2/output-1/build/libwebsockets-3.2.0/include/libwebsockets/lws-genhash.h:79:18: error: field 'ctx' has incomplete type
HMAC_CTX ctx;
^~~
This build failure is raised because openssl functions are not correcly
detected:
Determining if the function SSL_CTX_set1_param exists failed with the following output:
Change Dir: /home/buildroot/autobuild/instance-2/output-1/build/libwebsockets-3.2.0/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make cmTC_06946/fast && make[1]: Entering directory '/home/buildroot/autobuild/instance-2/output-1/build/libwebsockets-3.2.0/CMakeFiles/CMakeTmp'
/usr/bin/make -f CMakeFiles/cmTC_06946.dir/build.make CMakeFiles/cmTC_06946.dir/build
make[2]: Entering directory '/home/buildroot/autobuild/instance-2/output-1/build/libwebsockets-3.2.0/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_06946.dir/CheckFunctionExists.c.o
/home/buildroot/autobuild/instance-2/output-1/host/bin/arm-linux-gcc --sysroot=/home/buildroot/autobuild/instance-2/output-1/host/arm-buildroot-linux-uclibcgnueabihf/sysroot -DKEYWORD=__inline -Wall -Wsign-compare -Wuninitialized -Werror -Wundef -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -DCHECK_FUNCTION_EXISTS=SSL_CTX_set1_param -pthread -DNDEBUG -o CMakeFiles/cmTC_06946.dir/CheckFunctionExists.c.o -c /home/buildroot/autobuild/instance-2/output-1/host/share/cmake-3.15/Modules/CheckFunctionExists.c
Linking C executable cmTC_06946
/home/buildroot/autobuild/instance-2/output-1/host/bin/cmake -E cmake_link_script CMakeFiles/cmTC_06946.dir/link.txt --verbose=1
/home/buildroot/autobuild/instance-2/output-1/host/bin/arm-linux-gcc --sysroot=/home/buildroot/autobuild/instance-2/output-1/host/arm-buildroot-linux-uclibcgnueabihf/sysroot -Wall -Wsign-compare -Wuninitialized -Werror -Wundef -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -DCHECK_FUNCTION_EXISTS=SSL_CTX_set1_param -pthread -DNDEBUG CMakeFiles/cmTC_06946.dir/CheckFunctionExists.c.o -o cmTC_06946 /home/buildroot/autobuild/instance-2/output-1/host/arm-buildroot-linux-uclibcgnueabihf/sysroot/usr/lib/libssl.so /home/buildroot/autobuild/instance-2/output-1/host/arm-buildroot-linux-uclibcgnueabihf/sysroot/usr/lib/libcrypto.so -lssl -lcrypto -lm -lcap
/home/buildroot/autobuild/instance-2/output-1/host/opt/ext-toolchain/bin/../lib/gcc/arm-buildroot-linux-uclibcgnueabihf/8.3.0/../../../../arm-buildroot-linux-uclibcgnueabihf/bin/ld: cannot find -lpthread
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_06946.dir/build.make:88: recipe for target 'cmTC_06946' failed
Fixes:
- http://autobuild.buildroot.org/results/6186b4718db285edadf7203d00ed72f8d76a31e4
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
This changes the approach of tx credit management to set the
initial stream tx credit window to zero. This is the only way
with RFC7540 to gain the ability to selectively precisely rx
flow control incoming streams.
At the time the headers are sent, a WINDOW_UPDATE is sent with
the initial tx credit towards us for that specific stream. By
default, this acts as before with a 256KB window added for both
the stream and the nwsi, and additional window management sent
as stuff is received.
It's now also possible to set a member in the client info
struct and a new option LCCSCF_H2_MANUAL_RXFLOW to precisely
manage both the initial tx credit for a specific stream and
the ongoing rate limit by meting out further tx credit
manually.
Add another minimal example http-client-h2-rxflow demonstrating how
to force a connection's peer's initial budget to transmit to us
and control it during the connection lifetime to restrict the amount
of incoming data we have to buffer.
With light-on-dark terminal color schemes, 'black bold' (i.e. [30;1m) for
LLL_USER is illegible. I think this would be better as 'default bold' (i.e. [0;1m)
This should be a NOP for h2 support and only affects internal
apis. But it lets us reuse the working and reliable h2 mux
arrangements directly in other protocols later, and share code
so building for h2 + new protocols can take advantage of common
mux child handling struct and code.
Break out common mux handling struct into its own type.
Convert all uses of members that used to be in wsi->h2 to wsi->mux
Audit all references to the members and break out generic helpers
for anything that is useful for other mux-capable protocols to
reuse wsi->mux related features.
Saw this on travis selftests during context destroy
==18895== Invalid read of size 8
==18895== at 0x415909: __lws_vhost_destroy2 (vhost.c:1063)
==18895== by 0x40E65B: lws_context_destroy2 (context.c:929)
==18895== by 0x40EBE5: lws_context_destroy (context.c:1128)
==18895== by 0x40CC41: main (minimal-http-client-post.c:267)
==18895== Address 0x6168688 is 728 bytes inside a block of size 792 free'd
==18895== at 0x4C2BDEC: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==18895== by 0x45B29E: _realloc (alloc.c:120)
==18895== by 0x45B2D6: lws_realloc (alloc.c:130)
==18895== by 0x415ED7: __lws_vhost_destroy2 (vhost.c:1204)
==18895== by 0x419164: lws_vhost_unbind_wsi (wsi.c:82)
==18895== by 0x41236B: __lws_free_wsi (close.c:154)
==18895== by 0x4134CF: __lws_close_free_wsi_final (close.c:650)
==18895== by 0x4133BA: __lws_close_free_wsi (close.c:610)
==18895== by 0x413528: lws_close_free_wsi (close.c:660)
==18895== by 0x4158C7: __lws_vhost_destroy2 (vhost.c:1053)
==18895== by 0x40E65B: lws_context_destroy2 (context.c:929)
==18895== by 0x40EBE5: lws_context_destroy (context.c:1128)
Removing the last wsi from the vhost we started to destroy finalized the
vhost destruction, which is aimed at libuv async close cleanup. But if
we already entered __lws_vhost_destroy2, we will definitely destroy the vhost
ourselves at the end of that function already. So defeat the wsi close
triggering it.
If a client connects to a SSL server and the server sends handshake
alert (e.g. no matching ciphers) SSL_connect() fails, but because
SSL_ERROR_SSL return value is not handled, it's not considered a
failure. SSL_want_read() will return 1 and the client will happily wait
for more data from the server. Now if the server closes connection after
sending handshake alert, POLLIN event will be triggered,
lws_tls_client_connect() called again, but SSL_connect() will fail
without calling read(), so the client will end up consuming 100% CPU
because POLLIN will be triggered repeatedly.
Similar error handling is used in lws_tls_server_accept() and the
condition checks for SSL_ERROR_SSL. Using the same condition in
lws_tls_client_connect() fixes the problem.
Tested with OpenSSL 1.0.2k.
On some platforms, it's possible that logging flow may reset errno. In the case where
we try to log errno on those platforms and afterwards try to query it, we will get a
nasty surprise that the logged errno is destroyed by the time we come to test it.
In the two cases of this in the tree at the moment, sample errno into a temp and
log and test the temp.
Thanks to Sakthi Ramabadran for finding this.
Now the generic lws_system blobs can cover client certs + key, let's
add support for applying one of the blob sets to a specific client
connection (rather than doing it via the vhost).
