Although RSA can be used directly for signing / JWS
on large chunks of data since it's only operating on
the hash, when JWE support arrives, which allows bulk
encryption, it's going to be mandatory to support
secondary AES ciphers to use on the bulk data.
This adds generic support for all AES modes that OpenSSL
and mbedTLS have in common, works on both mbedTLS and
OpenSSL the same, and adds unit tests for each mode
in api-test-gencrypto, to run in CI.
Until now the JOSE pieces only had enough support for ACME.
This patch improves the JWK parsing to prepare for more
complete support and for adding JWE, genaes and genec in
later patches.
Add generic http compression layer eanbled at cmake with LWS_WITH_HTTP_STREAM_COMPRESSION.
This is wholly a feature of the HTTP role (used by h1 and h2 roles) and doesn't exist
outside that context.
Currently provides 'deflate' and 'br' compression methods for server side only.
'br' requires also -DLWS_WITH_HTTP_BROTLI=1 at cmake and the brotli libraries (available in
your distro already) and dev package.
Other compression methods can be added nicely using an ops struct.
The built-in file serving stuff will use this is the client says he can handle it, and the
mimetype of the file either starts with "text/" (html and css etc) or is the mimetype of
Javascript.
zlib allocates quite a bit while in use, it seems to be around 256KiB per stream. So this
is only useful on relatively strong servers with lots of memory. However for some usecases
where you are serving a lot of css and js assets, it's a nice help.
The patch performs special treatment for http/1.1 pipelining, since the compression is
performed on the fly the compressed content-length is not known until the end. So for h1
only, chunked transfer-encoding is automatically added so pipelining can continue of the
connection.
For h2 the chunking is neither supported nor required, so it "just works".
User code can also request to add a compression transform before the reply headers were
sent using the new api
LWS_VISIBLE int
lws_http_compression_apply(struct lws *wsi, const char *name,
unsigned char **p, unsigned char *end, char decomp);
... this allows transparent compression of dynamically generated HTTP. The requested
compression (eg, "deflate") is only applied if the client headers indicated it was
supported, otherwise it's a NOP.
Name may be NULL in which case the first compression method in the internal table at
stream.c that is mentioned as acceptable by the client will be used.
NOTE: the compression translation, same as h2 support, relies on the user code using
LWS_WRITE_HTTP and then LWS_WRITE_HTTP_FINAL on the last part written. The internal
lws fileserving code already does this.
Several new ops are planned for tls... so better to bite the bullet and
clean it out to the same level as roles + event-libs first.
Also adds a new travis target "mbedtls" and all the tests except
autobahn against mbedtls build.
Since new roles may be incompatible with http, add support for
alpn names at the role struct, automatic generation of the
default list of alpn names that servers advertise, and the
ability to override the used alpn names per-vhost and per-
client connection.
This not only lets you modulate visibility or use of h2,
but also enables vhosts that only offer non-http roles,
as well as restricting http role vhosts to only alpn
identifiers related to http roles.
This adds h2 http support for the client api.
The public client api requires no changes, it will detect by
ALPN if the server can handle http/2, if so, it will use it.
Multiple client connections using the lws api will be mapped on
to the same single http/2 + tls socket using http/2 streams
that are serviced simultaneously where possible.
This just supports "http/1.1" alpn and lets the client know it
can use keepalive earlier if it affirmitively negotiated
"http/1.1" on alpn.
mbedTLS wrapper needed a small adaptation to also allow
per-client-ssl control of the alpn negotiation list.
AG: unlike openssl, mbedtls does not load the system trust store.
So this change will make client tls operations that work OK on openssl fail on
mbedtls unless you provide the correct CA cert.
This allows lws to distinguish between untrusted CAs, hostname
mismatches, expired certificates.
NOTE: LCCSCF_ALLOW_SELFSIGNED actually allows for untrusted CAs, and
will also skip hostname verification. This is somewhat a limitiation of
the current lws verification process.
AG: improve error reporting up to the CLIENT_CONNECTION_ERROR argument
and add a note specific to mbedtls in the test client. Adapt the test
client to note the CA requirement if built with mbedTLS. Adapt the
minimal test clients to have the CAs available and use them if mbedTLS.
LWIP_SOCKET_OFFSET is now nonzero, which I handled a while back.
But the C api support for it is broken in esp-idf.
select() takes unmodified fds, but FD_SET / FD_ISSET etc must have the
offset subtracted on their args.
With this we are working on current HEAD esp-idf.
1) Introduce LWS_WITH_GCOV to build with gcc / clang coverage instrumentation.
$ cd build
$ make clean && rm -f `find . -name "*.gcno" -o -name "*.gcda"` && make -j16 && sudo make install && sudo /usr/local/bin/libwebsockets-test-server -s
...
$ gcov `find . -name *.c.gcno | grep -v test-apps` -b | sed "/\.h.\$/,/^$/d"
The above are available in two helper scripts
- scripts/build-gcov.sh
- scripts/gcov.sh
2)
CMake defaults changed:
- LWS_WITH_ZIP_FOPS: OFF
- LWS_WITH_RANGES: OFF
- LWS_WITHOUT_EXTENSIONS: ON
- LWS_WITH_ZLIB: OFF
New CMake controls that default-OFF:
- LWS_WITH_GENHASH
- LWS_WITH_GENRSA
these are implied by LWS_WITH_JWS (which is implied by LWS_WITH_ACME)
3) rename ./lib/tls/XXX/server.c and client.c to XXX-server.c / XXX-client.c.
This is because gcov dumps its results using the .c filename part only,
the copies overwrite each other if there are different .c files in the tree
with the same filename part.
4) Add onetime test-client mode and test to ./test-apps/attack.sh
5) Add gcov howto in READMEs/README.build.md using attack.sh
Under the condition the associated client went away (turn off WLAN at the
client), we can spin forever mistaking NET_SEND_FAILED for WANT_WRITE,
which was also true. This makes sure we understand that was fatal
immediately.
This adds support for a plugin that can be attached to a vhost
to acquire and maintain its TLS cert automatically.
It works the same with both OpenSSL and mbedTLS backends, but
they can't share auth keys, delete the 'auth.jwk' file as it is
in the example JSON when switching between libs
Check for LWS_SERVER_OPTION_PEER_CERT_NOT_REQUIRED...
AG: Fix missing stanza needed to confirm client cert needed at all
Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>
This adds a single api on lws that allows querying elements from the
peer certificate on a connection.
The api works the same regardless of the TLS backend.
Like the simple hashes, these work the same regardless of if the backend is
OpenSSL or mbedTLS.
Also move into ./lib/tls/ and split into two specific to mbedtls or openssl
backends.
1) The original wrapper logic for accounting for partial
sends on mbedtls is broken... adapt it
2) mbedtls has an additional restriction you MUST come
back to retry with EXACTLY the unsent part that you
originally asked for, or it loses coherency in the TLS
tunnel.
3) the wrapper is inconsistent between read WANT state
dynamically reported from mbedtls apis and the
SSL_want() api results. Check both on read.
