Introduce a very lightweight html5 + css2.1+ stateful stream parser, along
the same lines as the lws json and cbor ones.
This is interesting primarily because of just how low-resource it is for
modest css + html, it uses an lwsac to hold the entirity of the css in
memory at once but the html is parsed in chunks without any need to keep
previous chunks around (chunks may be as small as 1 byte).
A user callback receives element entry and exit callbacks with payload and
all attributes parsed out, CSS related to the active element stack is
parsed to provide a list of active css attributes, which takes heap for the
duration of the parsing.
In effect this provides rich information about the html and css state to
the callback, which has the job of producing the layout in a user-defined
way.
As such, there is no DOM in memory at one time, there is only a stack of
active elements like <html><body><div>xxx with their associated attributes
(like class). So as it is, it does not support DOM modification such as
JS changing elements after parsing, although elements with interesting IDs
could be kept around by the callback. There is a corresponding tiny and
relatively flat heap usage regardless of html size.
Default CSS is specified as recommended in the CSS 2.1 standard.
Inline <style></style> elements are supported, but not pre-html5 style= in
element attributes, since these are incompatible with strict CSP.
What the attributes should mean on your system, eg, font-size, font-family
etc is left for the user callback to decide, along with how to lay out the
items using the CSS attributes, and render them.
Fixed point 32.32 constants are used (fraction expressed at parts in 100M)
instead of floating point.
If you have presentation needs, even on a constrained display on a
constrained microcontroller, this makes it feasible to use standardized
markup and styling instead of roll your own.
This adds optional display list support to lws_display, using DLOs (Display
List Objects). DLOs for rectangle / rounded rectangle (with circle as the
degenerate case), PNGs, JPEG and compressed, antialiased bitmapped fonts
and text primitives are provided.
Logical DLOs are instantiated on heap and listed into an lws_display_list
owner, DLOs handle attributes like position, bounding box, colour +
opacity, and local error diffusion backing buffer.
When the display list is complete, it can be rasterized a line at a time,
with scoped error diffusion resolved, such that no allocation for the
framebuffer is required at any point. DLOs are freed as the rasterization
moves beyond their bounding box.
Adds a platform registry binding names and other metadata to lws_display
fonts / PNGs / JPEGs. Provides registration, destruction and best match
selection apis.
Introduce a rewritten picojpeg that is able to operate statefully and
rasterize into an internal line ringbuffer, emitting a line of pixels
at a time to the caller. This is the JPEG equivalent of the lws
PNG decoder.
JPEG is based around 8- or 16- line height MCU blocks, depending on
the chroma coding, mandating a corresponding internal line buffer
requirement.
Example total heap requirement for various kinds of 600px width jpeg
decoding:
Grayscale: 6.5KB
RGB 4:4:4: 16.4KB
RGB 4:2:2v: 16.4KB
RGB 4:4:2h: 31KB
RGB 4:4:0: 31KB
No other allocations occur during decode.
Stateful stream parsing means decode can be paused for lack of input
at any time and resumed seamlessly when more input becomes available.
Add a rewritten version of upng that decodes statefully line by line, and so
does not require a bitmap buffer for the output. This compares to original
upng approach that needs heap allocations for the input, the whole output
and intermediate allocations.
Instead of buffers for input, decompression and output, it only allocates
2 x lines of RGBA pixels (ie, a few KB), and 32KB of decompressed data for
backward references in the decoder, and decodes as needed into the 2-line
buffer to produce line rasterized results. For a 600px width PNG, this is
just 40KB heap for the duration.
This introduces a fixed precision signed 32.32 fractional type that can
work on devices without an FPU.
The integer part works as an int32_t, the fractional part represents the
fractional proportion expressed as part of 100M, so 8 fractional decimal
digit precision which is more than enough for many applications.
Add and Sub are reasonably fast as they are scaled on to a combined
uint64_t, Multiply is a little slower as it takes four uint64_t multiplies
that are summed, and divide is expensive but accurate, done bitwise taking
up to 32 iterations involving uint64_t div and mod.
This adds apis that enable usage of compressed backtraces in heap
instrumentation.
A decompressor tool is also provided that emits a textual
call stack suitable for use with addr2line.
Just like there is a default protocol provided if none is specified that is
suitable for handling http GET, introduce a default SS policy that is also
suitable for the case of http GET where the user trusts the CA using the
ssl library or system trust store.
This adds an optional feature LEJP_FLAG_FEAT_OBJECT_INDEXES that changes
lejp to treat { } items as indexable in ctx->i[] / ctx->ipos, since they
also can take commas.
This may break existing uses so it requires the default-off feature flag to
enable it. The flags context field is zeroed by lejp_construct(), so any
flags should be set on ctx->flags after alling that.
There's also a flag LEJP_FLAG_LATEST available as an alias to enable any
desirable but not-backwards-compatible behaviour, including this.
Add the info to the README and adapt the unit test to do it both with and
without the FEAT_OBJECT_INDEXES flag.
lws_sequencer and lws_abstract were both false starts trying to do the
functionality of secure streams.
Since Secure Streams does a better job for both and there are no known
out-of-tree users of them, let's remove them and focus on Secure Streams.
This provides very memory-efficient CBOR stream parsing
and writing.
The parser converts pieces of CBOR into callbacks that define
the structure and collate string and blobs into buffer chunks
for extensible and easy access.
It is fragementation-safe and does not need all the CBOR in
the same place at one time, chunks of CBOR are parsed and
discarded as provided.
It does not allocate and just needs a few hundred bytes of
stack for even huge CBOR objects. Huge strings and blobs
are handled without needing memory to hold them atomically.
Includes ./minimal-examples/api-tests/api-test-lecp that
unit tests it against 82 official example CBORs and
26 additional test vectors from COSE (just checking the CBOR
parsing).
The writing apis allow printf style semantics with a variety
of CBOR-aware %-formats. The apis write into a context that
manages output buffer usage, if the output buffer fills,
then the apis return with an AGAIN code that lets you issue
and reset the output buffer and repeat the api all to issue
more output. The subsequent calls can occur much later or
from a different function context, so this is perfect for
WRITEABLE-mediated output from the network parts of lws.
See ./READMEs/README.cbor-lecp.md
Add 9 fault injection cases in SS creation flow, and 5 of those
instantiate in the minimal examples ctests. The other 4 relate
to static policy and server, I tested the server ones by hand.
These tests confirm the recent change to unpick create using
lws_ss_destroy.
This adds an api allowing fault injection path implementations to get hold
of pseudo-random numbers between an externally-provided range.
You can set it using, eg, --fault-injection "f1(10%),f1_delay(123..456)"
while f1 shows how to decide whether to inject the fault and f1_delay
provides a pseudo-random number between the two values for the fault
implementation code to use.
Add support for dynamically determining the CAs needed to validate server
certificates. This allows you to avoid instantiating > 120 X.509 trusted
CA certs and have them take up heap the whole time.
Works for both openssl and mbedtls.
See READMEs/README.jit-trust.md for the documentation
You likely want the next patch for http redirect enhancements as well.
This provides a build option LWS_WITH_CONMON that lets user code recover
detailed connection stats on client connections with the LCCSCF_CONMON
flag.
In addition to latencies for dns, socket connection, tls and first protocol
response where possible, it also provides the user code an unfiltered list
of DNS responses that the client received, and the peer it actually
succeded to connect to.
There are a few build options that are trying to keep and report
various statistics
- DETAILED_LATENCY
- SERVER_STATUS
- WITH_STATS
remove all those and establish a generic rplacement, lws_metrics.
lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.
Openmetrics export is supported, for, eg, prometheus scraping.
Update libressl rules for options api type,
add some docs to build, use CHECK_SYMBOL_EXISTS since
CHECK_FUNCTION_EXISTS is fooled by the conditionals
in the headers
This adds some new objects and helpers for keeping and logging
info on grouped allocations, a group is, eg, SS handles or client
wsis.
Allocated objects get a context-unique "tag" string intended to replace
%p / wsi pointers etc. Pointers quickly become confusing when
allocations are freed and reused, the tag string won't repeat
until you produce 2^64 objects in a context.
In addition the tag string documents the object group, with prefixes
like "wsi-" or "vh-" and contain object-specific additional
information like the vhost name, address / port or the role of the wsi.
At creation time the lws code can use a format string and args
to add whatever group-specific info makes sense, eg, a wsi bound
to a secure stream can also append the guid of the secure stream,
it's copied into the new object tag and so is still available
cleanly after the stream is destroyed if the wsi outlives it.
This creates a role for RFC3549 Netlink monitoring.
If the OS supports it (currently, linux) then each pt creates a wsi
with the netlink role and dumps the current routing table at pt init.
It then maintains a cache of the routing table in each pt.
Upon routing table changes an SMD message is issued as an event, and
Captive Portal Detection is triggered.
All of the pt's current connections are reassessed for routability under
the changed routing table, those that no longer have a valid route or
gateway are closed.
