This adds some new objects and helpers for keeping and logging
info on grouped allocations, a group is, eg, SS handles or client
wsis.
Allocated objects get a context-unique "tag" string intended to replace
%p / wsi pointers etc. Pointers quickly become confusing when
allocations are freed and reused, the tag string won't repeat
until you produce 2^64 objects in a context.
In addition the tag string documents the object group, with prefixes
like "wsi-" or "vh-" and contain object-specific additional
information like the vhost name, address / port or the role of the wsi.
At creation time the lws code can use a format string and args
to add whatever group-specific info makes sense, eg, a wsi bound
to a secure stream can also append the guid of the secure stream,
it's copied into the new object tag and so is still available
cleanly after the stream is destroyed if the wsi outlives it.
A few different places want to create wsis and basically repeat their
own versions of the flow. Let's unify it into one helper in wsi.c
Also require the context lock held (this only impacts LWS_MAX_SMP > 1)
If getaddrinfo() is not able to reach the server, there may be
a connectivity problem downstream of the device that has not
been recognized by the Captive Portal Detect pieces yet.
If it looks like that might have happened, used the getaddrinfo()
return to provoke a new CPD scan.
With synthetic tests, we can have an h1 connection open to a server
and ask for an h2-specific connection to the same thing... lws will
bind it to the idle h1 connection since the endpoint and tls matches.
This also makes it check that the alpn filtering matches h1 before
allowing that.
Fix an assumption about h2 being around if h1 is that crept in.
Add a sai scenario to catch this kind of problem, only needs one
build since testing lws' own consistency... add WITH_MINIMAL_EXAMPLES
as well
role ops are usually only sparsely filled, there are currently 20
function pointers but several roles only fill in two. No single
role has more than 14 of the ops. On a 32/64 bit build this part
of the ops struct takes a fixed 80 / 160 bytes then.
First reduce the type of the callback reason part from uint16_t to
uint8_t, this saves 12 bytes unconditionally.
Change to a separate function pointer array with a nybble index
array, it costs 10 bytes for the index and a pointer to the
separate array, for 32-bit the cost is
2 + (4 x ops_used)
and for 64-bit
6 + (8 x ops_used)
for 2 x ops_used it means 32-bit: 10 vs 80 / 64-bit: 22 vs 160
For a typical system with h1 (9), h2 (14), listen (2), netlink (2),
pipe (1), raw_skt (3), ws (12), == 43 ops_used out of 140, it means
the .rodata for this reduced from 32-bit: 560 -> 174 (386 byte
saving) and 64-bit: 1120 -> 350 (770 byte saving)
This doesn't account for the changed function ops calling code, two
ways were tried, a preprocessor macro and explicit functions
For an x86_64 gcc 10 build with most options, release mode,
.text + .rodata
before patch: 553282
accessor macro: 552714 (568 byte saving)
accessor functions: 553674 (392 bytes worse than without patch)
therefore we went with the macros
RFC6724 defines an ipv6-centric DNS result sorting algorithm, that
takes route and source address route information for the results
given by the DNS resolution, and sorts them in order of preferability,
which defines the order they should be tried in.
If LWS_WITH_NETLINK, then lws takes care about collecting and monitoring
the interface, route and source address information, and uses it to
perform the RFC6724 sorting to re-sort the DNS before trying to make
the connections.
This creates a role for RFC3549 Netlink monitoring.
If the OS supports it (currently, linux) then each pt creates a wsi
with the netlink role and dumps the current routing table at pt init.
It then maintains a cache of the routing table in each pt.
Upon routing table changes an SMD message is issued as an event, and
Captive Portal Detection is triggered.
All of the pt's current connections are reassessed for routability under
the changed routing table, those that no longer have a valid route or
gateway are closed.
This adds a helper to test if an sa46 is on an sa46-based subnet.
The compare helper is adapted to say that non INET/INET6 addresses with
the same AF match.
If we connect out to an IP address, or we adopt a connected socket,
from now on we want to hold the peer sockaddr in the wsi.
Adapt ACCESS_LOG to use this new copy rather than keep the
stringified version.
The cmake define isn't exported, and msvc objects to void * comparision to
non void *.
Also dump the callback pointer with any errors, since after looking it up
in your mapfile this is usually enough to understand which sul and when it
was set.
With SMP + event lib, extra locking is required when dealing with cross-thread
adoption case, and cross-vhost cases like wsi close, we need to hold the pt or
context lock.
These lock apis are NOPs when LWS_MAX_SMP == 1 which is the default.
This lets you find out the SMP Thread Service Index (tsi) that a wsi
is bound to. This allows you to, eg, filter a global wsi list so
you can find the ones that exist in your service thread context.
They have been in lib/roles/http for historical reasons, and all
ended up in client-handshake.c that doesn't describe what they
actually do any more. Separate out the staged client connect
related stage functions into
lib/core-net/client/client2.c: lws_client_connect_2_dnsreq()
lib/core-net/client/client3.c: lws_client_connect_3_connect()
lib/core-net/client/client4.c: lws_client_connect_4_established()
Move a couple of other functions from there that don't belong out to
tls-client.c and client-http.c, which is related to http and remains
in the http role dir.
Teach lws how to deal with date: and retry-after:
Add quick selftest into apt-test-lws_tokenize
Expand lws_retry_sul_schedule_retry_wsi() to check for retry_after and
increase the backoff if a larger one found.
Finally, change SS h1 protocol to handle 503 + retry-after: as a
failure, and apply any increased backoff from retry-after
automatically.
EXTERNAL_POLL is not recommended for use for a while, it's a hack to allow
integration of lws with random application poll() loops.
While lws is very happy to do that secondary job for any event lib using the
foreign loop support (for uv, event, glib, and ev), for random roll-your-
own poll() waits there's no api because there's no event lib. The solution
with a future is upgrade your application to use an event loop.
The test app that supports EXTERNAL_POLL was broken in Apr 2018, so it's
apparently good news nobody has been using it in new implementations
since then. This patch adds in the missing pieces so we can test it until
it is formally deprecated.
In the case http client doesn't get a response and closes, currently
it is confused, it reports it as a CLIENT_CONNECTION_ERROR but then
also a CLOSED_CLIENT_HTTP.
Adapt the logic so we only go that way for ws connection... not getting
the server headers means not reaching ESTABLISHED, which makes it a
CCE not a CLOSE.
Also make sure we never issue a CLOSE type callback if we issued a CCE.
For server, if the adoption of the incoming connection proceeds but then
fails early on, eg, tls alert due to hostname mismatch with cert, the
wsi close happens but it doesn't clean up the invalidated reference to
itself in the server ss object... if it became established, that's handled
by the ss protocol callback.
This patch helps the close path to understand there is a related ss object
and to clean up after itself.
This is complicated by the fact extern on a function declaration implies
visibility... we have to make LWS_EXTERN empty when building static.
And, setting target_compile_definitions() doesn't work inside macros,
so it has to be set explicitly for the plugins.
Checking the symbol status needs nm -C -D as per
https://stackoverflow.com/questions/37934388/symbol-visibility-not-working-as-expected
after this patch, libwebsockets.a shows no symbols when checked like that and
the static-linked minimal examples only show -U for their other dynamic
imports.
In a handful of cases we use LWS_EXTERN on extern data declarations,
those then need to change to explicit extern.
Event lib support as it has been isn't scaling well, at the low level
libevent and libev headers have a namespace conflict so they can't
both be built into the same image, and at the distro level, binding
all the event libs to libwebsockets.so makes a bloaty situation for
packaging, lws will drag in all the event libs every time.
This patch implements the plan discussed here
https://github.com/warmcat/libwebsockets/issues/1980
and refactors the event lib support so they are built into isolated
plugins and bound at runtime according to what the application says
it wants to use. The event lib plugins can be packaged individually
so that only the needed sets of support are installed (perhaps none
of them if the user code is OK with the default poll() loop). And
dependent user code can mark the specific event loop plugin package
as required so pieces are added as needed.
The eventlib-foreign example is also refactored to build the selected
lib support isolated.
A readme is added detailing the changes and how to use them.
https://libwebsockets.org/git/libwebsockets/tree/READMEs/README.event-libs.md
Move the common plugin scanning dir stuff to be based on lws_dir, which
already builds for windows. Previously this was done via dirent for unix
and libuv for windows.
Reduce the dl plat stuff to just wrap instantiation and destruction of
dynlibs, establish common code in lib/misc/dir.c for plugin scanning
itself.
Migrate the libuv windows dl stuff to windows-plugins.c, so that he's
available even if later libuv loop support becomes and event lib plugin.
Remove the existing api exports scheme for plugins, just export a const struct
now which has a fixed header type but then whatever you want afterwards depending
on the class / purpose of the plugin. Place a "class" string in the header so
there can be different kinds of plugins implying different types exported.
Make the plugin apis public and add support for filter by class string, and
per instantation / destruction callbacks so the subclassed header type can
do its thing for the plugin class. The user provides a linked-list base
for his class of plugins, so he can manage them completely separately and
in user code / user export types.
Rip out some last hangers-on from generic sessions / tables.
This is all aimed at making the plugins support general enough so it can
provide event lib plugins later.
