1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-09 00:00:04 +01:00
Commit graph

472 commits

Author SHA1 Message Date
Andy Green
4b089788bc tls: mbedtls-3
Adapt mbedtls support for compatibility with v3, while maintaining
compatibility with v2.

Notice v3 has removed the ability to encrypt with pubkey and
decrypt with privkey.  Openssl still has it, atm with v3 these
fall back to encrypt with privkey and decrypt with pubkey.

> The RSA module no longer supports private-key operations with the
> public key or vice versa. As a consequence, RSA operation functions
> no longer have a mode parameter. If you were calling RSA operations
> with the normal mode (public key for verification or encryption,
> private key for signature or decryption), remove the
> MBEDTLS_MODE_PUBLIC or MBEDTLS_MODE_PRIVATE argument. If you were
> calling RSA operations with the wrong mode, which rarely makes sense
>from a security perspective, this is no longer supported.
2021-07-13 13:22:50 +01:00
Andy Green
4c4b2c5dcf libevent: use event_enable_debug_mode 2021-07-13 08:27:18 +01:00
Bing Zhao
c604b9cb21 tls: add option to serialize ssl handshake
This is useful for memory constrained systems such as esp32
to improve heap usage minfree/watermark.

On esp32 with simultaneous_ssl_restriction = 2 (MQTT+H1),
Without this change, sys heap minfree ranges from 59k to 71k
  in 100 iterations, average 66.5k.
With this change, sys heap minfree ranges from 64k to 71k
  in 100 interations, average 68.7k
2021-07-08 06:22:47 +01:00
Andy Green
fabe78d222 Wextra
Add -Wextra (with -Wno-unused-parameter) to unix builds in addition to
-Wall -Werror.

This can successfully build everything in Sai without warnings / errors.
2021-07-04 10:29:54 +01:00
Andy Green
5432e8ae7b sspc: add concurrency stress
Add a way to perform sspc tests with increased concurrency and sequential
tests
2021-07-04 10:29:52 +01:00
Andy Green
8e76634ed3 logs: introduce log_cx 2021-07-01 05:20:53 +01:00
Andy Green
604968a31d minimal: hcmulti: modernize startup to use OPERATIONAL 2021-06-27 08:42:28 +01:00
Jed Lu
c2e10db5f2 ss-policy2c-update 2021-06-26 12:31:44 +01:00
Andy Green
1c72e9eee6 minimal: ws client spam: modernize with pvo and cancel service after interrupted 2021-06-24 16:55:25 +01:00
Andy Green
272dba8307 http: redo redirects to follow close flow
Until now although we can follow redirects, and they can promote the
protocol from h1->h2, we couldn't handle h2 wsi reuse since there are many
states in the wsi affected by being h2.

This wipes the related states in lws_wsi_reset() and follows the generic
wsi close flow before deviating into the redirect really close to the end,
ensuring we cleaned out evidence of our previous life properly.

h2->h2 redirects work properly after this.

The max number of redirects is increased from 3 -> 4 since this was seen in
the wild with www and then geographic-based redirects.
2021-06-22 15:55:29 +01:00
Andy Green
2f9bb7a30a tls: JIT Trust
Add support for dynamically determining the CAs needed to validate server
certificates.  This allows you to avoid instantiating > 120 X.509 trusted
CA certs and have them take up heap the whole time.

Works for both openssl and mbedtls.

See READMEs/README.jit-trust.md for the documentation

You likely want the next patch for http redirect enhancements as well.
2021-06-22 15:55:29 +01:00
Andy Green
983edafdb4 mbedtls: add AKID and SKID support 2021-06-22 15:55:29 +01:00
Andy Green
b25079c4b4 lws_cache_ttl 2021-06-22 15:55:29 +01:00
Andy Green
c09d69e291 minimal: htt-client: send user agent and accept
Some servers will 4xx us if they don't see these headers
2021-06-22 15:54:13 +01:00
Andy Green
3ef0b0268d ctest: do not reuse ctest-ssp path 2021-06-22 15:54:05 +01:00
Jed Lu
2abf4115a8 ss: direct protocol string 2021-06-11 14:44:10 +01:00
Jed Lu
e82778e07a ss: static policy for sigv4 2021-06-11 14:25:12 +01:00
Andy Green
9fe4b37650 wilcard: change api to use length for check
The wilcard part was always length driven... but the check part relied on
being NUL terminated.  Change that to also use a length specifier and not
need any NUL.

To migrate old uses, just add a fourth param as strlen(check).
2021-06-07 14:57:57 +01:00
Andy Green
509d0528a9 examples: embedded: keep loop running on WROVER 2021-06-07 08:17:51 +01:00
Andy Green
7baf400017 evlib: allow custom evlib ops
Adapt the event lib support slighly so we can pass an event lib "plugin"
header in at context creation time, and direct all event loop handling to
go via that.

This can then be lightly adapted to interface to an existing custom event
loop cleanly, without the problems of EXTERNAL_POLL.

The external loop must consult with us about the max wait timeout as shown
in the added minimal-http-server-eventlib-custom example.

The example shows a complete implementation working with a custom poll()
loop cleanly while only needing 5 ops in the custom event lib handler.
2021-06-07 08:17:49 +01:00
Andy Green
0517a2a629 ss: split out blob into own minimal example
The --blob option requires GENCRYPTO that's not on by default, to handle
the hash checks... that's going to cause a lot of confusion because it
means the simplest ss example won't build by default then.

Let's remove the blob support (and GENCRYPTO dependency) from the simplest
example and make a new minimal-secure-streams-blob example that has --blob
support and the GENCRYPTO dependency as well.
2021-05-28 10:10:15 +01:00
Andy Green
ad3aa5339e TCP_FASTOPEN 2021-05-25 08:13:13 +01:00
Or Lupovitz
16328b1918 mbedtls: server: enforce client cert check when vhost requires 2021-05-23 14:58:07 +01:00
Catalin Raceanu
cf867faa46 tls: client: session cache synthetic cb 2021-04-29 20:03:41 +01:00
Yichen Gu
5fdd9c99b1 ss: policy2c: type error on respmap 2021-04-21 19:30:42 +01:00
Andy Green
891f28b43e ss: proxy: fix conn deref on onward
Trying to use the opaque pointer in the handle to point to the conn isn't
going to work when we need it to point to the ss handle.

Move it to have its on place in the handle.
2021-04-21 10:02:00 +01:00
Andy Green
aa6b3f418e mbedtls: allow central trust store preload
Mbedtls does not have the same concept as openssl about preloading the
system trust store into every SSL_CTX.

This patch allows you to simulate the behaviour by passing in a context
creation-time filepath that all client SSL_CTX will be initialized from.
2021-04-15 19:10:54 +01:00
Andy Green
68d9f3a7f2 ss: wire up EVENT_WAIT_CANCELLED
Currently the lws_cancel_service() api only manifests itself at lws level.
This adds a state LWSSSCS_EVENT_WAIT_CANCELLED that is broadcast to all
SS in the event loop getting the cancel service api call, and allows
SS-level user code to pick up handling events from other threads.

There's a new example minimal-secure-streams-threads which shows the
pattern for other threads to communicate with and trigger the event in the
lws service thread.
2021-04-15 19:10:54 +01:00
Andy Green
5b6a89f79d ss: add test for OS trust store mdoe
It's already the case that leaving off the "tls_trust_store" member of the
streamtype definition in the policy causes the streamtype to validate its
tls connections via the OS trust store, usually a bundle OpenSSL has been
configured to load at init automagically, but also literally the OS trust
store in windows case.

Add tests to confirm that.
2021-04-15 19:10:54 +01:00
Andy Green
24abd699f6 http: parser: straighten out %00 legality
https://github.com/warmcat/libwebsockets/issues/2262

This adds a README explaining what can be expected if your URLs contain
%00, and adds a safe helper for urlargs-by-name that is length-based.

Contains fix for extra NUL on some headers

https://github.com/warmcat/libwebsockets/issues/2267
2021-04-10 09:21:33 +01:00
Andy Green
42dc817d8f ss: proxy: get rx flow control working
This fixes the proxy rx flow by adding an lws_dsh helper to hide the
off-by-one in the "kind" array (kind 0 is reserved for tracking the
unallocated dsh blocks).

For testing, it adds a --blob option on minimal-secure-streams[-client]
which uses a streamtype "bulkproxflow" from here

https://warmcat.com/policy/minimal-proxy-v4.2-v2.json

		"bulkproxflow": {
			"endpoint": "warmcat.com",
			"port": 443,
			"protocol": "h1",
			"http_method": "GET",
			"http_url": "blob.bin",
			"proxy_buflen": 32768,
			"proxy_buflen_rxflow_on_above": 24576,
			"proxy_buflen_rxflow_off_below": 8192,
			"tls": true,
			"retry": "default",
			"tls_trust_store": "le_via_dst"
		}

This downloads a 51MB blob of random data with the SHA256sum

ed5720c16830810e5829dfb9b66c96b2e24efc4f93aa5e38c7ff4150d31cfbbf

The minimal-secure-streams --blob example client delays the download by
50ms every 10KiB it sees to force rx flow usage at the proxy.

It downloads the whole thing and checks the SHA256 is as expected.

Logs about rxflow status are available at LLL_INFO log level.
2021-04-07 15:54:26 +01:00
Andy Green
d7ce068543 ctest: ss: minimal-hugeurl: run correct test 2021-04-05 11:06:54 +01:00
Andy Green
37160c797e dsh: api-test: dont use lws_dsh_describe in Release 2021-04-05 10:55:04 +01:00
Andy Green
d291c02a23 ss: sspc: add conmon performance telemetry
This provides a way to get ahold of LWS_WITH_CONMON telemetry from Secure
Streams, it works the same with direct onward connections or via the proxy.

You can mark streamtypes with a "perf": true policy attribute... this
causes the onward connections on those streamtypes to collect information
about the connection performance, and the unsorted DNS results.

Streams with that policy attribute receive extra data in their rx callback,
with the LWSSS_FLAG_PERF_JSON flag set on it, containing JSON describing the
performance of the onward connection taken from CONMON data, in a JSON
representation.  Streams without the "perf" attribute set never receive
this extra rx.

The received JSON is based on the CONMON struct info and looks like

{"peer":"46.105.127.147","dns_us":596,"sockconn_us":31382,"tls_us":28180,"txn_resp_us:23015,"dns":["2001:41d0:2:ee93::1","46.105.127.147"]}

A new minimal example minimal-secure-streams-perf is added that collects
this data on an HTTP GET from warmcat.com, and is built with a -client
version as well if LWS_WITH_SECURE_STREAMS_PROXY_API is set, that operates
via the ss proxy and produces the same result at the client.
2021-04-05 10:55:04 +01:00
Andy Green
51490ae6e6 Fault injection
add lws_xos: xoshiro256 PRNG
2021-04-05 10:55:04 +01:00
Andy Green
7451702722 tls-sessions: serialization 2021-04-04 05:45:53 +01:00
Catalin
d5753b6298 tls: openssl-only: client-only: session caching 2021-04-04 05:34:11 +01:00
Pavel Otchertsov
f84b90ba03 use gmtime_r instead of gmtime if possible
Also check tm result is not NULL before using it.
2021-03-25 10:08:40 +00:00
Pavel Otchertsov
b5ed38395e use ctime_r instead of ctime if possible 2021-03-25 10:08:38 +00:00
Andreas Weigel
319a00c067 jwt: sign via info
Offer a more extensible way to form and sign JWTs
2021-03-25 08:40:52 +00:00
Andy Green
f7da3217f9 evlib-foreign: add client connection after context creation 2021-03-25 08:40:50 +00:00
Andy Green
0f42f0d9ad uloop 2021-03-17 12:23:34 +00:00
Andy Green
06509e287d lws_conmon: connection monitoring and stats generation
This provides a build option LWS_WITH_CONMON that lets user code recover
detailed connection stats on client connections with the LCCSCF_CONMON
flag.

In addition to latencies for dns, socket connection, tls and first protocol
response where possible, it also provides the user code an unfiltered list
of DNS responses that the client received, and the peer it actually
succeded to connect to.
2021-03-17 07:31:49 +00:00
Andy Green
1000a52df3 logs: prioritize WITH_NO_LOGS even in release
Really not having any logs makes it difficult to know what is really
happening, but if that's you're thing this will align debug and release
modes to just have ERR and USER if you give WITH_NO_LOGS
2021-03-15 12:56:15 +00:00
Andy Green
546c151555 ss: add set_metadata that takes a heap copy and make ss-h1 use it
Until now we set metadata value pointers into the onward wsi ah data
area... that's OK until we get a situation the wsi has gone away before we
have a chance to deliver the metadata over the proxy link.

Add a variant lws_ss_alloc_set_metadata() that allocates space on the heap
and takes a copy of the input metadata.  Change ss-h1 to alloc copies of
its metadata so we no longer race the wsi ah lifetime.
2021-03-14 15:05:18 +00:00
Andy Green
4804624905 ss: force check all set_metadata returns
lws_ss_set_metadata can fail... eg, due to transient OOM situation... if it does,
caller must take appropriate action like disconnect and retry.

So mark the api as requiring the result checking, and make sure all the
examples do it.
2021-03-14 08:58:26 +00:00
Andy Green
9d13b97e37 client: DNS failure should retry while waiting for connect timeout
If the DNS lookup fails, we just sit out the remaining connect time.

The adapts it to reuse the wsi->sul_connect_timeout to schedule DNS lookup
retries until we're out of time.

Eventually we want to try other things as well, this is aligned with that.

Found with fault injection.
2021-03-11 06:58:59 +00:00
Andy Green
3f4623bb36 lws_metrics
There are a few build options that are trying to keep and report
various statistics

 - DETAILED_LATENCY
 - SERVER_STATUS
 - WITH_STATS

remove all those and establish a generic rplacement, lws_metrics.

lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.

Openmetrics export is supported, for, eg, prometheus scraping.
2021-03-08 21:47:28 +00:00
Andy Green
b6aa4c2d5d solaris: pthread is not a ptr 2021-03-08 20:44:10 +00:00
Andy Green
ede7f8b0f3 ss: support huge urls 2021-03-05 14:13:26 +00:00