Event lib support as it has been isn't scaling well, at the low level
libevent and libev headers have a namespace conflict so they can't
both be built into the same image, and at the distro level, binding
all the event libs to libwebsockets.so makes a bloaty situation for
packaging, lws will drag in all the event libs every time.
This patch implements the plan discussed here
https://github.com/warmcat/libwebsockets/issues/1980
and refactors the event lib support so they are built into isolated
plugins and bound at runtime according to what the application says
it wants to use. The event lib plugins can be packaged individually
so that only the needed sets of support are installed (perhaps none
of them if the user code is OK with the default poll() loop). And
dependent user code can mark the specific event loop plugin package
as required so pieces are added as needed.
The eventlib-foreign example is also refactored to build the selected
lib support isolated.
A readme is added detailing the changes and how to use them.
https://libwebsockets.org/git/libwebsockets/tree/READMEs/README.event-libs.md
Currently we always reserve a fakewsi per pt so events that don't have a related actual
wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks
that look reasonable to user protocol handler code expecting a valid wsi every time.
This patch splits out stuff that user callbacks often unconditionally expect to be in
a wsi, like context pointer, vhost pointer etc into a substructure, which is composed
into struct lws at the top of it. Internal references (struct lws is opaque, so there
are only internal references) are all updated to go via the substructre, the compiler
should make that a NOP.
Helpers are added when fakewsi is used and referenced.
If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before,
although the helpers improve consistency by zeroing down the substructure. There is
a huge amount of user code out there over the last 10 years that did not always have
the minimal examples to follow, some of it does some unexpected things.
If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of
being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't
reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then
create a struct lws_a (the substructure) on the stack, zero it down (but it is only
like 4 pointers) and prepare it with whatever we know like the context.
Then we cast it to a struct lws * and use it in the user protocol handler call.
In this case, the remainder of the struct lws is undefined. However the amount of
old protocol handlers that might touch things outside of the substructure in
PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is
significant on constrained devices.
User handlers should not be touching everything in a wsi every time anyway, there
are several cases where there is no valid wsi to do the call with. Dereference of
things outside the substructure should only happen when the callback reason shows
there is a valid wsi bound to the activity (as in all the minimal examples).
If the client connection attempt fails early, we report it will a NULL
return from the client connection api. If it fails later, perhaps after
more times around the event loop, we report it as a CONNECTION_ERROR.
This patch makes sure we don't do CONNECTION_ERROR flow if in fact we
are still in the client_connect_via_info() and in a position to report
the failure by returning NULL from there, without it under some
conditions we will do both a CONNECTION_ERROR and return NULL.
- Add low level system message distibution framework
- Add support for local Secure Streams to participate using _lws_smd streamtype
- Add apit test and minimal example
- Add SS proxy support for _lws_smd
See minimal-secure-streams-smd README.md
Adapt the pt sul owner list to be an array, and define two different lists,
one that acts like before and is the default for existing users, and another
that has the ability to cooperate with systemwide suspend to restrict the
interval spent suspended so that it will wake in time for the earliest
thing on this wake-suspend sul list.
Clean the api a bit and add lws_sul_cancel() that only needs the sul as the
argument.
Add a flag for client creation info to indicate that this client connection
is important enough that, eg, validity checking it to detect silently dead
connections should go on the wake-suspend sul list. That flag is exposed in
secure streams policy so it can be added to a streamtype with
"swake_validity": true
Deprecate out the old vhost timer stuff that predates sul. Add a flag
LWS_WITH_DEPRECATED_THINGS in cmake so users can get it back temporarily
before it will be removed in a v4.2.
Adapt all remaining in-tree users of it to use explicit suls.
Establish a new distributed CMake architecture with CMake code related to
a source directory moving to be in the subdir in its own CMakeLists.txt.
In particular, there's now one in ./lib which calls through to ones
further down the directory tree like ./lib/plat/xxx, ./lib/roles/xxx etc.
This cuts the main CMakelists.txt from 98KB -> 33KB, about a 66% reduction,
and it's much easier to maintain sub-CMakeLists.txt that are in the same
directory as the sources they manage, and conceal all the details that that
level.
Child CMakelists.txt become responsible for:
- include_directories() definition (this is not supported by CMake
directly, it passes it back up via PARENT_SCOPE vars in helper
macros)
- Addition child CMakeLists.txt inclusion, for example toplevel ->
role -> role subdir
- Source file addition to the build
- Dependent library path resolution... this is now a private thing
in the child CMakeLists.txt, it just passes back any adaptations
to include_directories() and the LIB_LIST without filling the
parent namespace with the details
Until now if the generic transaction completes and the connection is idle,
there's a fixed 5s grace period to keep the connection up in case something
else wants it.
This allows setting it in the client creation info struct .keep_warm_secs.
If left at 0, then it maintains the backward-compatible 5s wait.
Secure Streams is an optional layer on top of lws that separates policy
like endpoint selection and tls cert validation into a device JSON
policy document.
Code that wants to open a client connection just specifies a streamtype name,
and no longer deals with details like the endpoint, the protocol (!) or anything
else other than payloads and optionally generic metadata; the JSON policy
contains all the details for each streamtype. h1, h2, ws and mqtt client
connections are supported.
Logical secure streams outlive any particular connection and supports "nailed-up"
connectivity regardless of underlying connection stability.
Adds client support for MQTT QoS0 and QoS1, compatible with AWS IoT
Supports stream binding where independent client connections to the
same endpoint can mux on a single tcp + tls connection with topic
routing managed internally.
There are some minor public api type improvements rather than cast everywhere
inside lws and user code to work around them... these changed from int to
size_t
- lws_buflist_use_segment() return
- lws_tokenize_t .len and .token_len
- lws_tokenize_cstr() length
- lws_get_peer_simple() namelen
- lws_get_peer_simple_fd() namelen, int fd -> lws_sockfd_type fd
- lws_write_numeric_address() len
- lws_sa46_write_numeric_address() len
These changes are typically a NOP for user code
This changes the approach of tx credit management to set the
initial stream tx credit window to zero. This is the only way
with RFC7540 to gain the ability to selectively precisely rx
flow control incoming streams.
At the time the headers are sent, a WINDOW_UPDATE is sent with
the initial tx credit towards us for that specific stream. By
default, this acts as before with a 256KB window added for both
the stream and the nwsi, and additional window management sent
as stuff is received.
It's now also possible to set a member in the client info
struct and a new option LCCSCF_H2_MANUAL_RXFLOW to precisely
manage both the initial tx credit for a specific stream and
the ongoing rate limit by meting out further tx credit
manually.
Add another minimal example http-client-h2-rxflow demonstrating how
to force a connection's peer's initial budget to transmit to us
and control it during the connection lifetime to restrict the amount
of incoming data we have to buffer.
Now the generic lws_system blobs can cover client certs + key, let's
add support for applying one of the blob sets to a specific client
connection (rather than doing it via the vhost).
This shouldn't be necessary; just END_HEADERS flag should be enough.
But nghttp2 will not talk to us unless we end the stream from our side.
Unfortunately ending the stream at the time we sent the headers means
we cannot support the long poll half-close scheme. So add a quirk
flag to optionally support this behaviour of nghttp2 when the client
is creating the connection.
Introduce a generic lws_state object with notification handlers
that may be registered in a chain.
Implement one of those in the context to manage the "system state".
Allow other pieces of lws and user code to register notification
handlers on a context list. Handlers can object to or take over
responsibility to move forward and retry system state changes if
they know that some dependent action must succeed first.
For example if the system time is invalid, we cannot move on to
a state where anything can do tls until that has been corrected.
Refactor everything around ping / pong handling in ws and h2, so there
is instead a protocol-independent validity lws_sul tracking how long it
has been since the last exchange that confirms the operation of the
network connection in both directions.
Clean out periodic role callback and replace the last two role users
with discrete lws_sul for each pt.
Remove LWS_LATENCY.
Add the option LWS_WITH_DETAILED_LATENCY, allowing lws to collect very detailed
information on every read and write, and allow the user code to provide
a callback to process events.
Improve the code around stash, getting rid of the strdups for a net
code reduction. Remove the special destroy helper for stash since
it becomes a one-liner.
Trade several stack allocs in the client reset function for a single
sized brief heap alloc to reduce peak stack alloc by around 700 bytes.
External poll support generates a lot of messages on a busy system
for no value unless you're one of the few people using it. It's
not recommended for new users and is there for backwards compatibility.
Make it not built by default and selectable by cmake option.
Generic sessions has been overdue some love to align it with
the progress in the rest of lws.
1) Strict Content Security Policy
2) http2 compatibility
3) fixes and additions for use in a separate process via unix domain socket
4) work on ws and http proxying in lws
5) add minimal example
