Refactor everything around ping / pong handling in ws and h2, so there
is instead a protocol-independent validity lws_sul tracking how long it
has been since the last exchange that confirms the operation of the
network connection in both directions.
Clean out periodic role callback and replace the last two role users
with discrete lws_sul for each pt.
Old certs were getting near the end of their life and we switched the
server to use letsencrypt. The root and intermediate needed for the
mbedtls case changed accordingly
wsi timeout, wsi hrtimer, sequencer timeout and vh-protocol timer
all now participate on a single sorted us list.
The whole idea of polling wakes is thrown out, poll waits ignore the
timeout field and always use infinite timeouts.
Introduce a public api that can schedule its own callback from the event
loop with us resolution (usually ms is all the platform can do).
Upgrade timeouts and sequencer timeouts to also be able to use us resolution.
Introduce a prepared fakewsi in the pt, so we don't have to allocate
one on the heap when we need it.
Directly handle vh-protocol timer if LWS_MAX_SMP == 1
There are quite a few linked-lists of things that want events after
some period. This introduces a type binding an lws_dll2 for the
list and a lws_usec_t for the duration.
The wsi timeouts, the hrtimer and the sequencer timeouts are converted
to use these, also in the common event wait calculation.
Travis seems to be restricting the number of outgoing connections
or the rate of them... we have been using 10 concurrent and 100 connections
[2019/08/02 09:26:22:7950] USER: callback_minimal_spam: established (try 10, est 8, closed 0, err 0)
[2019/08/02 09:26:22:8041] USER: callback_minimal_spam: established (try 10, est 9, closed 0, err 0)
[2019/08/02 09:26:23:0098] USER: callback_minimal_spam: reopening (try 11, est 10, closed 1, err 0)
[2019/08/02 09:26:23:0105] USER: callback_minimal_spam: reopening (try 12, est 10, closed 2, err 0)
[2019/08/02 09:26:23:0111] USER: callback_minimal_spam: reopening (try 13, est 10, closed 3, err 0)
[2019/08/02 09:26:23:0117] USER: callback_minimalRROR: closed before established (try 25, est 14, closed 14, err 2)
[2019/08/02 09:26:44:6125] ERR: CLIENT_CONNECTION_ERROR: closed before established (try 26, est 14, closed 14, err 3)
[2019/08/02 09:26:44:6129] ERR: CLIENT_CONNECTION_ERROR: closed before established (try 27, est 14, closed 14, err 4)
[2019/08/02 09:26:44:6133] ERR: CLIENT_CONNECTION_ERROR: closed before established (try 28, est 14, closed 14, err 5)
[2019/08/02 09:26:44:6137] ERR: CLIENT_CONNECTION_ERROR: closed before established (try 29, est 14, closed 14, err 6)
[2019/08/02 09:26:45:6152] ERR: CLIENT_CONNECTION_ERROR: closed before established (try 30, est 14, closed 14, err 7)
[2019/08/02 09:26:45:6163] ERR: CLIENT_CONNECTION_ERROR: closed before established (try 31, est 14, closed 14, err 8)
[2019/08/02 09:26:45:6168] ERR: CLIENT_CONNECTION_ERROR: closed before established (try 32, est 14, closed 14, err 9)
[2019/08/02 09:26:45:6174] ERR: CLIENT_CONNECTION_ERROR: closed before established (try 33, est 14, closed 14, err 10)
[2019/08/02 09:26:47:0635] USER: callback_minimal_spam: established (try 34, est 14, closed 14, err 10)
Reduce to 3 concurrent / 15 see if it helps travis get over the hump
The logic in the loops for insertion and deletion from the
mini, forced to non ulimit max fds in the pt mode was not
quite right.
It showed up in hard to reproduce problem with the ws client
spam test that uses the mini mode, on travis. This should
fix the root cause.
An lws context usually contains a processwide fd -> wsi lookup table.
This allows any possible fd returned by a *nix type OS to be immediately
converted to a wsi just by indexing an array of struct lws * the size of
the highest possible fd, as found by ulimit -n or similar.
This works modestly for Linux type systems where the default ulimit -n for
a process is 1024, it means a 4KB or 8KB lookup table for 32-bit or
64-bit systems.
However in the case your lws usage is much simpler, like one outgoing
client connection and no serving, this represents increasing waste. It's
made much worse if the system has a much larger default ulimit -n, eg 1M,
the table is occupying 4MB or 8MB, of which you will only use one.
Even so, because lws can't be sure the OS won't return a socket fd at any
number up to (ulimit -n - 1), it has to allocate the whole lookup table
at the moment.
This patch looks to see if the context creation info is setting
info->fd_limit_per_thread... if it leaves it at the default 0, then
everything is as it was before this patch. However if finds that
(info->fd_limit_per_thread * actual_number_of_service_threads) where
the default number of service threads is 1, is less than the fd limit
set by ulimit -n, lws switches to a slower lookup table scheme, which
only allocates the requested number of slots. Lookups happen then by
iterating the table and comparing rather than indexing the array
directly, which is obviously somewhat of a performance hit.
However in the case where you know lws will only have a very few wsi
maximum, this method can very usefully trade off speed to be able to
avoid the allocation sized by ulimit -n.
minimal examples for client that can make use of this are also modified
by this patch to use the smaller context allocations.
https://libwebsockets.org/pipermail/libwebsockets/2019-April/007937.html
thanks to Bruce Perens for noting it.
This doesn't change the intention or status of the CC0 files, they were
pure CC0 before (ie, public domain) and they are pure CC0 now. It just
gets rid of the (C) part at the top of the dedication which may be read
to be a bit contradictory since the purpose is to make it public domain.
During client redirect we "reset" the wsi to the redirect address,
involving closing the current fd that was told to redirect (it will
usually be a completely different server or port).
With libuv and its two-stage close that's not trivial. This solves
the problem we will "reset" (overwrite) where the handle lives in the
wsi with new a new connection / handle by having it copied out into
an allocated watcher struct, which is freed in the uv close callback.
To confirm it the minimal ws client example gets some new options, the
original problem was replicated with this
$ lws-minimal-ws-client-echo -s invalid.url.com -p 80
https://github.com/warmcat/libwebsockets/issues/1390
You can build lws without support for ws, with -DLWS_ROLE_WS=0.
This is thanks to the role ops isolating all the ws-specific business
in the ws role.
Also retire more test apps replaced by minmal-examples.
This replaces the old test-app for echo with separate client and server
minimal versions.
The autobahn test script is made more autonomous and tests both
client and server.
This converts several of the selftests to return a status in their exit code
about whether they 'worked'.
A small bash script framework is added, with a selftest.sh in the mininmal
example dirs that support it, and a ./minimal-examples/selftests.sh script
that can be run from the build dir with no args that discovers and runs all
the selftest.sh scripts underneath.
That is also integrated into travis and the enabled tests must pass now for
travis to pass. Travis does not have a modern libuv so it can't run a
couple of tests which are nulled out if it sees it's running in travis env.
Since new roles may be incompatible with http, add support for
alpn names at the role struct, automatic generation of the
default list of alpn names that servers advertise, and the
ability to override the used alpn names per-vhost and per-
client connection.
This not only lets you modulate visibility or use of h2,
but also enables vhosts that only offer non-http roles,
as well as restricting http role vhosts to only alpn
identifiers related to http roles.
This adds h2 http support for the client api.
The public client api requires no changes, it will detect by
ALPN if the server can handle http/2, if so, it will use it.
Multiple client connections using the lws api will be mapped on
to the same single http/2 + tls socket using http/2 streams
that are serviced simultaneously where possible.
AG: unlike openssl, mbedtls does not load the system trust store.
So this change will make client tls operations that work OK on openssl fail on
mbedtls unless you provide the correct CA cert.
This allows lws to distinguish between untrusted CAs, hostname
mismatches, expired certificates.
NOTE: LCCSCF_ALLOW_SELFSIGNED actually allows for untrusted CAs, and
will also skip hostname verification. This is somewhat a limitiation of
the current lws verification process.
AG: improve error reporting up to the CLIENT_CONNECTION_ERROR argument
and add a note specific to mbedtls in the test client. Adapt the test
client to note the CA requirement if built with mbedTLS. Adapt the
minimal test clients to have the CAs available and use them if mbedTLS.
This adds an lws cmake option that builds all the minimal examples as part of lws,
it's useful for QA.
It adds a macro to examples that depend on a particular lws configuration to understand
they should just null out their project definition in builds where the lws configuration
requirement is not met, and we are building as part of lws.
It also adapts all the example library additions to select the just-built-but-not-yet-installed
library in the case it is built as part of lws. If built standalone, it now uses the cmake
platform-abstracted way to add the library requirement too.
