This provides very memory-efficient CBOR stream parsing
and writing.
The parser converts pieces of CBOR into callbacks that define
the structure and collate string and blobs into buffer chunks
for extensible and easy access.
It is fragementation-safe and does not need all the CBOR in
the same place at one time, chunks of CBOR are parsed and
discarded as provided.
It does not allocate and just needs a few hundred bytes of
stack for even huge CBOR objects. Huge strings and blobs
are handled without needing memory to hold them atomically.
Includes ./minimal-examples/api-tests/api-test-lecp that
unit tests it against 82 official example CBORs and
26 additional test vectors from COSE (just checking the CBOR
parsing).
The writing apis allow printf style semantics with a variety
of CBOR-aware %-formats. The apis write into a context that
manages output buffer usage, if the output buffer fills,
then the apis return with an AGAIN code that lets you issue
and reset the output buffer and repeat the api all to issue
more output. The subsequent calls can occur much later or
from a different function context, so this is perfect for
WRITEABLE-mediated output from the network parts of lws.
See ./READMEs/README.cbor-lecp.md
Add support for dynamically determining the CAs needed to validate server
certificates. This allows you to avoid instantiating > 120 X.509 trusted
CA certs and have them take up heap the whole time.
Works for both openssl and mbedtls.
See READMEs/README.jit-trust.md for the documentation
You likely want the next patch for http redirect enhancements as well.
Mbedtls does not have the same concept as openssl about preloading the
system trust store into every SSL_CTX.
This patch allows you to simulate the behaviour by passing in a context
creation-time filepath that all client SSL_CTX will be initialized from.
This provides a build option LWS_WITH_CONMON that lets user code recover
detailed connection stats on client connections with the LCCSCF_CONMON
flag.
In addition to latencies for dns, socket connection, tls and first protocol
response where possible, it also provides the user code an unfiltered list
of DNS responses that the client received, and the peer it actually
succeded to connect to.
There are a few build options that are trying to keep and report
various statistics
- DETAILED_LATENCY
- SERVER_STATUS
- WITH_STATS
remove all those and establish a generic rplacement, lws_metrics.
lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.
Openmetrics export is supported, for, eg, prometheus scraping.
openssl v3-alpha11 has marked EC_KEY pieces as deprecated... we use it in
LWS_WITH_GENCRYPTO but the related RSA etc pieces were already deprecated
for that. We use EC_KEY pieces in vhost init...
The apis are not removed but deprecated, we should have a way to keep
trucking, but as it is the deprecation warning is promoted to an error.
Let's add LWS_SUPPRESS_DEPRECATED_API_WARNINGS option off by default. If
enabled at cmake, external deprecated api warnings are suppressed. This
gives a general workaround for now for opensslv3.
In addition, even if you don't do that, let's notice we are on openssl v3
and don't build the EC curve selection stuff, I don't think anyone is
actually using it anyway.
The elaborated tags for ls owsi, vh, and ss objects are very useful
signposts to understand what's happening in the logs. But for busy
h1 servers, they're just a lot of noise.
This lets you stop the logging of tagged object lifecycle by cmake
-DLWS_LOG_TAG_LIFECYCLE=0
Add the ability to just build plugins into the main library.
They are already designed to have a pinhole export for when
they are used as dynamic lib plugins so their namespace
does not conflict.
This is a huge patch that should be a global NOP.
For unix type platforms it enables -Wconversion to issue warnings (-> error)
for all automatic casts that seem less than ideal but are normally concealed
by the toolchain.
This is things like passing an int to a size_t argument. Once enabled, I
went through all args on my default build (which build most things) and
tried to make the removed default cast explicit.
With that approach it neither change nor bloat the code, since it compiles
to whatever it was doing before, just with the casts made explicit... in a
few cases I changed some length args from int to size_t but largely left
the causes alone.
From now on, new code that is relying on less than ideal casting
will complain and nudge me to improve it by warnings.
C++ APIs wrapping SS client
These are intended to provide an experimental protocol-independent c++
api even more abstracted than secure streams, along the lines of
"wget -Omyfile https://example.com/thing"
WIP
RFC6724 defines an ipv6-centric DNS result sorting algorithm, that
takes route and source address route information for the results
given by the DNS resolution, and sorts them in order of preferability,
which defines the order they should be tried in.
If LWS_WITH_NETLINK, then lws takes care about collecting and monitoring
the interface, route and source address information, and uses it to
perform the RFC6724 sorting to re-sort the DNS before trying to make
the connections.
This creates a role for RFC3549 Netlink monitoring.
If the OS supports it (currently, linux) then each pt creates a wsi
with the netlink role and dumps the current routing table at pt init.
It then maintains a cache of the routing table in each pt.
Upon routing table changes an SMD message is issued as an event, and
Captive Portal Detection is triggered.
All of the pt's current connections are reassessed for routability under
the changed routing table, those that no longer have a valid route or
gateway are closed.
The cmake define isn't exported, and msvc objects to void * comparision to
non void *.
Also dump the callback pointer with any errors, since after looking it up
in your mapfile this is usually enough to understand which sul and when it
was set.
Event lib support as it has been isn't scaling well, at the low level
libevent and libev headers have a namespace conflict so they can't
both be built into the same image, and at the distro level, binding
all the event libs to libwebsockets.so makes a bloaty situation for
packaging, lws will drag in all the event libs every time.
This patch implements the plan discussed here
https://github.com/warmcat/libwebsockets/issues/1980
and refactors the event lib support so they are built into isolated
plugins and bound at runtime according to what the application says
it wants to use. The event lib plugins can be packaged individually
so that only the needed sets of support are installed (perhaps none
of them if the user code is OK with the default poll() loop). And
dependent user code can mark the specific event loop plugin package
as required so pieces are added as needed.
The eventlib-foreign example is also refactored to build the selected
lib support isolated.
A readme is added detailing the changes and how to use them.
https://libwebsockets.org/git/libwebsockets/tree/READMEs/README.event-libs.md
- Add low level system message distibution framework
- Add support for local Secure Streams to participate using _lws_smd streamtype
- Add apit test and minimal example
- Add SS proxy support for _lws_smd
See minimal-secure-streams-smd README.md
The low level apis for HMAC (including those only introduced at 1.1.0...)
are all deprecated in OpenSSL v3.
Let's bite the bullet and migrate to EVP, it's already existing in modern
OpenSSL and we already use it for genhash.
EVP needs a PKEY, sort that out and keep it around until the hmac is
destroyed.
Esp-idf has an improved but still kind of abused cmake-
based build system now.
If we see ESP_PLATFORM coming as a cmake var, we can know we
are being built from inside the esp-idf config system.
Leave the existing esp32 arrangements alone but triggered off
ESP_PLATFORM, adapt to use the cross toolchain file and
various quirks automatically.
In this way you can build lws a part of your project in a
much cleaner way.
Prepare a minimal esp32 test app for use in Sai
Adapt .sai.json to build for esp32
Lws now strips out http headers releated to h2, ws and unusual headers
based on cmake config settings for those features... it saves some heap
for the ah and reduces the table size in .rodata.
It's possible code might have some external dependency on the original
header indexes, but, eg, you don't enable h2 so those indexes are
optimized with the h2 ones taken out.
This introduces a cmake option "LWS_HTTP_HEADERS_ALL", default-OFF, that
defeats the header table optimization for compatibility with older
versions in the case the client software can't be adapted to use the
lws-exported matching header enums.
You probably don't need this.
LWS builds OK on iOS SDK as unix type plat, except it
doesn't have net/route.h.
Detect we're building on iOS at CMake and export a
preprocessor define we can use to snip out the missing
include.
By default this doesn't change any existing logging behaviour at all.
But it allows you to define cmake options to force or force-disable the
build of individual log levels using new cmake option bitfields
LWS_LOGGING_BITFIELD_SET and LWS_LOGGING_BITFIELD_CLEAR.
Eg, -DLWS_LOGGING_BITFIELD_SET="(LLL_INFO)" can force INFO log level
built even in release mode. -DLWS_LOGGING_BITFIELD_CLEAR="(LLL_NOTICE)"
will likewise remove NOTICE logging from the build regardless of
DEBUG or RELEASE mode.
Add support for external pthreads lib on windows and some docs about how to do.
It can build with LWS_WITH_THREADPOOL and LWS_WITH_MINIMAL_EXAMPLES including the
pthreads-dependent ones without warnings or errors on windows platform as well with this.
pthreads_t can be anything, including a struct - not a pointer-to-a-struct
but the struct itself. These can't be cast to a void * for printing as they can
on linux, where the base type is a pointer.
Let's fix all the usage of those to determine their own thread index in terms
of the meaning to the program rather than as a tid.
In some cases devices may be too constrained to handle JSON policies but still
want to use SS apis and methodology.
This introduces an off-by-default cmake option LWS_WITH_SECURE_STREAMS_STATIC_POLICY_ONLY,
if enabled the JSON parsing part is excluded and it's assumed the user code
provides its policy as hardcoded policy structs.
Secure Streams is an optional layer on top of lws that separates policy
like endpoint selection and tls cert validation into a device JSON
policy document.
Code that wants to open a client connection just specifies a streamtype name,
and no longer deals with details like the endpoint, the protocol (!) or anything
else other than payloads and optionally generic metadata; the JSON policy
contains all the details for each streamtype. h1, h2, ws and mqtt client
connections are supported.
Logical secure streams outlive any particular connection and supports "nailed-up"
connectivity regardless of underlying connection stability.
Adds client support for MQTT QoS0 and QoS1, compatible with AWS IoT
Supports stream binding where independent client connections to the
same endpoint can mux on a single tcp + tls connection with topic
routing managed internally.
Headers related to ws or h2 are now elided if the ws or h2 role
is not enabled for build. In addition, a new build-time option
LWS_WITH_HTTP_UNCOMMON_HEADERS on by default allows removal of
less-common http headers to shrink the parser footprint.
Minilex is adapted to produce 8 different versions of the lex
table, chosen at build-time according to which headers are
included in the build.
If you don't need the unusual headers, or aren't using h2 or ws,
this chops down the size of the ah and the rodata needed to hold
the parsing table from 87 strings / pointers to 49, and the
parsing table from 1177 to 696 bytes.
