Adapt mbedtls support for compatibility with v3, while maintaining
compatibility with v2.
Notice v3 has removed the ability to encrypt with pubkey and
decrypt with privkey. Openssl still has it, atm with v3 these
fall back to encrypt with privkey and decrypt with pubkey.
> The RSA module no longer supports private-key operations with the
> public key or vice versa. As a consequence, RSA operation functions
> no longer have a mode parameter. If you were calling RSA operations
> with the normal mode (public key for verification or encryption,
> private key for signature or decryption), remove the
> MBEDTLS_MODE_PUBLIC or MBEDTLS_MODE_PRIVATE argument. If you were
> calling RSA operations with the wrong mode, which rarely makes sense
>from a security perspective, this is no longer supported.
Make sure MBEDTLS_INCLUDE_DIRS and MBEDTLS_LIBRARIES are set in parent CMakeLists
Also set MBEDTLS_INCLUDE_DIRS in CMAKE_REQUIRED_INCLUDES so the CHECK_C_SOURCE_COMPILES
checks actually evaluate properly.
lws_tls_restrict_borrow() returns error when tls restriction limit is
reached. However lws_ssl_close() still calls lws_tls_restrict_return()
to decrease simultaneous_ssl. Thus LWS accepts more than allowed ssl
links, making simultaneous_ssl_restriction useless.
Fix it by tracking lws_tls_restrict_borrow() return value and only
calling lws_tls_restrict_return() if lws_tls_restrict_borrow() is
successful.
Add -Wextra (with -Wno-unused-parameter) to unix builds in addition to
-Wall -Werror.
This can successfully build everything in Sai without warnings / errors.
Add support for dynamically determining the CAs needed to validate server
certificates. This allows you to avoid instantiating > 120 X.509 trusted
CA certs and have them take up heap the whole time.
Works for both openssl and mbedtls.
See READMEs/README.jit-trust.md for the documentation
You likely want the next patch for http redirect enhancements as well.
Mbedtls does not have the same concept as openssl about preloading the
system trust store into every SSL_CTX.
This patch allows you to simulate the behaviour by passing in a context
creation-time filepath that all client SSL_CTX will be initialized from.
There are a few build options that are trying to keep and report
various statistics
- DETAILED_LATENCY
- SERVER_STATUS
- WITH_STATS
remove all those and establish a generic rplacement, lws_metrics.
lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.
Openmetrics export is supported, for, eg, prometheus scraping.
On h2 server POST, there's a race to see if the POST body is going to be
received coalesced with the headers.
The problem is on h2, we can't action the stream http request or body until
the stream is writeable, since we may start issuing the response right away;
there's already DEFERRING_ACTION state to manage this. And indeed, the
coalesced, not-immediately-actionable POST body is buflisted properly.
However when we come to action the POST using buflisted data, we don't follow
the same pattern as dealing with the incoming data immediately.
This patch aligns the pattern dumping the buflist content to track
expected rx_content_length and handle BODY_COMPLETION if we got to
the end of it, along with removal from the pt list of wsi with pending
buflists if we used it up.
This is a huge patch that should be a global NOP.
For unix type platforms it enables -Wconversion to issue warnings (-> error)
for all automatic casts that seem less than ideal but are normally concealed
by the toolchain.
This is things like passing an int to a size_t argument. Once enabled, I
went through all args on my default build (which build most things) and
tried to make the removed default cast explicit.
With that approach it neither change nor bloat the code, since it compiles
to whatever it was doing before, just with the casts made explicit... in a
few cases I changed some length args from int to size_t but largely left
the causes alone.
From now on, new code that is relying on less than ideal casting
will complain and nudge me to improve it by warnings.
This adds some new objects and helpers for keeping and logging
info on grouped allocations, a group is, eg, SS handles or client
wsis.
Allocated objects get a context-unique "tag" string intended to replace
%p / wsi pointers etc. Pointers quickly become confusing when
allocations are freed and reused, the tag string won't repeat
until you produce 2^64 objects in a context.
In addition the tag string documents the object group, with prefixes
like "wsi-" or "vh-" and contain object-specific additional
information like the vhost name, address / port or the role of the wsi.
At creation time the lws code can use a format string and args
to add whatever group-specific info makes sense, eg, a wsi bound
to a secure stream can also append the guid of the secure stream,
it's copied into the new object tag and so is still available
cleanly after the stream is destroyed if the wsi outlives it.
mbedtls has older and newer hashing apis, if we detected the newer ones
we use them like the older ones since generally, hashing doesn't have a
way to fail for sha and md5 update etc.
Coverity objects... change it to dumbly check the retcode.
Some general debugging advice but also really clarify the official way of how to dump
what is going out and coming in directly from the tls tunnel, so you can see the
actual data unencrypted.
Currently we always reserve a fakewsi per pt so events that don't have a related actual
wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks
that look reasonable to user protocol handler code expecting a valid wsi every time.
This patch splits out stuff that user callbacks often unconditionally expect to be in
a wsi, like context pointer, vhost pointer etc into a substructure, which is composed
into struct lws at the top of it. Internal references (struct lws is opaque, so there
are only internal references) are all updated to go via the substructre, the compiler
should make that a NOP.
Helpers are added when fakewsi is used and referenced.
If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before,
although the helpers improve consistency by zeroing down the substructure. There is
a huge amount of user code out there over the last 10 years that did not always have
the minimal examples to follow, some of it does some unexpected things.
If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of
being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't
reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then
create a struct lws_a (the substructure) on the stack, zero it down (but it is only
like 4 pointers) and prepare it with whatever we know like the context.
Then we cast it to a struct lws * and use it in the user protocol handler call.
In this case, the remainder of the struct lws is undefined. However the amount of
old protocol handlers that might touch things outside of the substructure in
PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is
significant on constrained devices.
User handlers should not be touching everything in a wsi every time anyway, there
are several cases where there is no valid wsi to do the call with. Dereference of
things outside the substructure should only happen when the callback reason shows
there is a valid wsi bound to the activity (as in all the minimal examples).
Mbedtls mbedtls_x509_crt_parse() returns 0 for success which is good.
But it has a complicated idea about what to return on fail... if it
couldn't make even one cert from the data, then it returns a negative
return indicating the parsing problem and there is nothing to free.
If it managed to parse at least one cert, instead it retuns a positive
number indicating the number of certs it didn't parse successfully,
and there is something to free.
Adapt the code to understand this quirk.
LWS_EXTERN needs to be empty for windows when declaring functions in the
headers. But for data, it needs the explicit extern otherwise on windows
or mingw based builds, it thinks we are redeclaring the data each time.
Establish a new distributed CMake architecture with CMake code related to
a source directory moving to be in the subdir in its own CMakeLists.txt.
In particular, there's now one in ./lib which calls through to ones
further down the directory tree like ./lib/plat/xxx, ./lib/roles/xxx etc.
This cuts the main CMakelists.txt from 98KB -> 33KB, about a 66% reduction,
and it's much easier to maintain sub-CMakeLists.txt that are in the same
directory as the sources they manage, and conceal all the details that that
level.
Child CMakelists.txt become responsible for:
- include_directories() definition (this is not supported by CMake
directly, it passes it back up via PARENT_SCOPE vars in helper
macros)
- Addition child CMakeLists.txt inclusion, for example toplevel ->
role -> role subdir
- Source file addition to the build
- Dependent library path resolution... this is now a private thing
in the child CMakeLists.txt, it just passes back any adaptations
to include_directories() and the LIB_LIST without filling the
parent namespace with the details
Esp-idf has an improved but still kind of abused cmake-
based build system now.
If we see ESP_PLATFORM coming as a cmake var, we can know we
are being built from inside the esp-idf config system.
Leave the existing esp32 arrangements alone but triggered off
ESP_PLATFORM, adapt to use the cross toolchain file and
various quirks automatically.
In this way you can build lws a part of your project in a
much cleaner way.
Prepare a minimal esp32 test app for use in Sai
Adapt .sai.json to build for esp32
Replace the bash selftest plumbing with CTest.
To use the selftests, build with -DLWS_WITH_MINIMAL_EXAMPLES=1
and `CTEST_OUTPUT_ON_FAILURE=1 make test` or just
`make test`.
To disable tests that require internet access, also give
-DLWS_CTEST_INTERNET_AVAILABLE=0
Remove travis and appveyor scripts on master.
Remove travis and appveyor decals on README.md.
Nobody uses this from the wrapper... it brings in mbedtls_x509_crt_info()
even if you're using -ffunctionsections for a loss of about 1.7KB on Arm.
Let's chop it out...
The mbedtls openssl wrapper wants to use exports from mbedtls' net_sockets.c,
but this is only supposed to work on *nix and windows. Typically people
are using mbedtls on RTOS type platforms and to use it, net_sockets.c
needs some hacking.
Try to avoid that situation by porting the two exports we need into the
lws plat code and call from the wrapper.
Surprisingly -fdata-sections -ffunction-sections does not remove any string literals and __func__
implicit .rodata generated by the removed function's compilation.
That means potentially considerable deadweight is in the image even if the function is removed
at linktime.
