Long story, there's also minimal example to take care of.
This change should take care of anything with ETHER_ADDR_LEN in
net/ethernet.h plus windows, for both the lib and example.
Linux has a sockopt flag defined by RFC5014 that informs IPv6 systems with
SLAAC config to prefer to bind the socket to a public address instead of
any temporary private address.
This patch adds a client info flag LCCSCF_IPV6_PREFER_PUBLIC_ADDR that lets
the user indicate the client socket should be prepared with the public
address binding preference.
Currently it's only implemented on Linux.
Introduce a very lightweight html5 + css2.1+ stateful stream parser, along
the same lines as the lws json and cbor ones.
This is interesting primarily because of just how low-resource it is for
modest css + html, it uses an lwsac to hold the entirity of the css in
memory at once but the html is parsed in chunks without any need to keep
previous chunks around (chunks may be as small as 1 byte).
A user callback receives element entry and exit callbacks with payload and
all attributes parsed out, CSS related to the active element stack is
parsed to provide a list of active css attributes, which takes heap for the
duration of the parsing.
In effect this provides rich information about the html and css state to
the callback, which has the job of producing the layout in a user-defined
way.
As such, there is no DOM in memory at one time, there is only a stack of
active elements like <html><body><div>xxx with their associated attributes
(like class). So as it is, it does not support DOM modification such as
JS changing elements after parsing, although elements with interesting IDs
could be kept around by the callback. There is a corresponding tiny and
relatively flat heap usage regardless of html size.
Default CSS is specified as recommended in the CSS 2.1 standard.
Inline <style></style> elements are supported, but not pre-html5 style= in
element attributes, since these are incompatible with strict CSP.
What the attributes should mean on your system, eg, font-size, font-family
etc is left for the user callback to decide, along with how to lay out the
items using the CSS attributes, and render them.
Fixed point 32.32 constants are used (fraction expressed at parts in 100M)
instead of floating point.
If you have presentation needs, even on a constrained display on a
constrained microcontroller, this makes it feasible to use standardized
markup and styling instead of roll your own.
This adds optional display list support to lws_display, using DLOs (Display
List Objects). DLOs for rectangle / rounded rectangle (with circle as the
degenerate case), PNGs, JPEG and compressed, antialiased bitmapped fonts
and text primitives are provided.
Logical DLOs are instantiated on heap and listed into an lws_display_list
owner, DLOs handle attributes like position, bounding box, colour +
opacity, and local error diffusion backing buffer.
When the display list is complete, it can be rasterized a line at a time,
with scoped error diffusion resolved, such that no allocation for the
framebuffer is required at any point. DLOs are freed as the rasterization
moves beyond their bounding box.
Adds a platform registry binding names and other metadata to lws_display
fonts / PNGs / JPEGs. Provides registration, destruction and best match
selection apis.
Introduce a rewritten picojpeg that is able to operate statefully and
rasterize into an internal line ringbuffer, emitting a line of pixels
at a time to the caller. This is the JPEG equivalent of the lws
PNG decoder.
JPEG is based around 8- or 16- line height MCU blocks, depending on
the chroma coding, mandating a corresponding internal line buffer
requirement.
Example total heap requirement for various kinds of 600px width jpeg
decoding:
Grayscale: 6.5KB
RGB 4:4:4: 16.4KB
RGB 4:2:2v: 16.4KB
RGB 4:4:2h: 31KB
RGB 4:4:0: 31KB
No other allocations occur during decode.
Stateful stream parsing means decode can be paused for lack of input
at any time and resumed seamlessly when more input becomes available.
The adapted upng has a very compact fully-stateful lws-aligned
implementation already.
Adapt it to also be buildable and operable standalone, and to
understand gzip headers.
Provide some apis to inflate gzip simply reusing opaque inflator
contexts from upng.
Provide an api test that inflates gzip files from stdin -> stdout
Add a rewritten version of upng that decodes statefully line by line, and so
does not require a bitmap buffer for the output. This compares to original
upng approach that needs heap allocations for the input, the whole output
and intermediate allocations.
Instead of buffers for input, decompression and output, it only allocates
2 x lines of RGBA pixels (ie, a few KB), and 32KB of decompressed data for
backward references in the decoder, and decodes as needed into the 2-line
buffer to produce line rasterized results. For a 600px width PNG, this is
just 40KB heap for the duration.
This adds apis that enable usage of compressed backtraces in heap
instrumentation.
A decompressor tool is also provided that emits a textual
call stack suitable for use with addr2line.
This patch checks for the env var SSLKEYLOGFILE=path, if present, then
client connection tls secrets are appended into path.vhostname.
This allows decryption of captured encrypted data for debugging purposes.
SSKEYLOGFILE=path env var method is the same as provided by Firefox and
Chrome for this purpose.
Although many of the examples must be run from the example directory as
cwd, everyone getting started probably wants to try the examples, cmake
knows how to do it, so let's enable it by default.
For plugins that handle PROTOCOL_INIT, we have to iterate any PLUGINS_BUILTIN
plugins as if we had just discovered and loaded them from plugin files, so
they bind to vhosts appropriately.
Add some private helpers to keep the guts from duplication in plat.
This is a NOP for existing usecases.
At the moment the only implemented transport for serialized SS is wsi, it's
typically used with Unix Domain Sockets, but it also works over tcp the
same.
It generalizes the interface between serialized chunks and the
transport, separately for client and proxy. The wsi transport is migrated
to use the new transport ops structs.
It will then be possible to "bring your own transport", so long as it is
reliable, and in-order, both for proxy and client / sspc.
We also adapt minimal-secure-streams-binance to build the -client variant
via SS proxy as well.
LWS_ONLY_SSPC is added so libwebsockets can be produced with just sspc
client support even for tiny targets.
A new embedded minimal example for rpi pico is also provided that
demonstrates using Serialized SS over a UART to an SS proxy, to implement
the SS Binance example on the pico, even though it has no networking itself.
This provides very memory-efficient CBOR stream parsing
and writing.
The parser converts pieces of CBOR into callbacks that define
the structure and collate string and blobs into buffer chunks
for extensible and easy access.
It is fragementation-safe and does not need all the CBOR in
the same place at one time, chunks of CBOR are parsed and
discarded as provided.
It does not allocate and just needs a few hundred bytes of
stack for even huge CBOR objects. Huge strings and blobs
are handled without needing memory to hold them atomically.
Includes ./minimal-examples/api-tests/api-test-lecp that
unit tests it against 82 official example CBORs and
26 additional test vectors from COSE (just checking the CBOR
parsing).
The writing apis allow printf style semantics with a variety
of CBOR-aware %-formats. The apis write into a context that
manages output buffer usage, if the output buffer fills,
then the apis return with an AGAIN code that lets you issue
and reset the output buffer and repeat the api all to issue
more output. The subsequent calls can occur much later or
from a different function context, so this is perfect for
WRITEABLE-mediated output from the network parts of lws.
See ./READMEs/README.cbor-lecp.md
Add support for dynamically determining the CAs needed to validate server
certificates. This allows you to avoid instantiating > 120 X.509 trusted
CA certs and have them take up heap the whole time.
Works for both openssl and mbedtls.
See READMEs/README.jit-trust.md for the documentation
You likely want the next patch for http redirect enhancements as well.
Mbedtls does not have the same concept as openssl about preloading the
system trust store into every SSL_CTX.
This patch allows you to simulate the behaviour by passing in a context
creation-time filepath that all client SSL_CTX will be initialized from.
This provides a build option LWS_WITH_CONMON that lets user code recover
detailed connection stats on client connections with the LCCSCF_CONMON
flag.
In addition to latencies for dns, socket connection, tls and first protocol
response where possible, it also provides the user code an unfiltered list
of DNS responses that the client received, and the peer it actually
succeded to connect to.
There are a few build options that are trying to keep and report
various statistics
- DETAILED_LATENCY
- SERVER_STATUS
- WITH_STATS
remove all those and establish a generic rplacement, lws_metrics.
lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.
Openmetrics export is supported, for, eg, prometheus scraping.
openssl v3-alpha11 has marked EC_KEY pieces as deprecated... we use it in
LWS_WITH_GENCRYPTO but the related RSA etc pieces were already deprecated
for that. We use EC_KEY pieces in vhost init...
The apis are not removed but deprecated, we should have a way to keep
trucking, but as it is the deprecation warning is promoted to an error.
Let's add LWS_SUPPRESS_DEPRECATED_API_WARNINGS option off by default. If
enabled at cmake, external deprecated api warnings are suppressed. This
gives a general workaround for now for opensslv3.
In addition, even if you don't do that, let's notice we are on openssl v3
and don't build the EC curve selection stuff, I don't think anyone is
actually using it anyway.
The elaborated tags for ls owsi, vh, and ss objects are very useful
signposts to understand what's happening in the logs. But for busy
h1 servers, they're just a lot of noise.
This lets you stop the logging of tagged object lifecycle by cmake
-DLWS_LOG_TAG_LIFECYCLE=0
Add the ability to just build plugins into the main library.
They are already designed to have a pinhole export for when
they are used as dynamic lib plugins so their namespace
does not conflict.
This is a huge patch that should be a global NOP.
For unix type platforms it enables -Wconversion to issue warnings (-> error)
for all automatic casts that seem less than ideal but are normally concealed
by the toolchain.
This is things like passing an int to a size_t argument. Once enabled, I
went through all args on my default build (which build most things) and
tried to make the removed default cast explicit.
With that approach it neither change nor bloat the code, since it compiles
to whatever it was doing before, just with the casts made explicit... in a
few cases I changed some length args from int to size_t but largely left
the causes alone.
From now on, new code that is relying on less than ideal casting
will complain and nudge me to improve it by warnings.
C++ APIs wrapping SS client
These are intended to provide an experimental protocol-independent c++
api even more abstracted than secure streams, along the lines of
"wget -Omyfile https://example.com/thing"
WIP
RFC6724 defines an ipv6-centric DNS result sorting algorithm, that
takes route and source address route information for the results
given by the DNS resolution, and sorts them in order of preferability,
which defines the order they should be tried in.
If LWS_WITH_NETLINK, then lws takes care about collecting and monitoring
the interface, route and source address information, and uses it to
perform the RFC6724 sorting to re-sort the DNS before trying to make
the connections.
This creates a role for RFC3549 Netlink monitoring.
If the OS supports it (currently, linux) then each pt creates a wsi
with the netlink role and dumps the current routing table at pt init.
It then maintains a cache of the routing table in each pt.
Upon routing table changes an SMD message is issued as an event, and
Captive Portal Detection is triggered.
All of the pt's current connections are reassessed for routability under
the changed routing table, those that no longer have a valid route or
gateway are closed.
The cmake define isn't exported, and msvc objects to void * comparision to
non void *.
Also dump the callback pointer with any errors, since after looking it up
in your mapfile this is usually enough to understand which sul and when it
was set.
