1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-09 00:00:04 +01:00
Commit graph

68 commits

Author SHA1 Message Date
Andy Green
b71445eb43 lws_mux_mark_immortal: remove noisy log 2025-02-25 19:34:29 +00:00
Andy Green
9dbbd45170 lws_get_urlarg_by_name: drop candidates that wont fit in buf rather than bail
https://github.com/warmcat/libwebsockets/issues/3227
2024-09-28 06:03:06 +01:00
titofra
5ab6e04c79 lws_xxx_all_protocol: also compare by protocol names 2024-04-12 06:04:40 +01:00
Yichen Gu
87dd9283e1 wsimux: immortal: only process immortality once per stream
We can call mark immortal multiple times for a child stream, make sure it
is only processed (and increments the nwsi refcount) the once.
2021-11-09 07:45:58 +00:00
Andy Green
005e130393 h2: make has_buffered_out check nwsi 2021-09-13 15:15:31 +01:00
Andy Green
ee510154a0 fi: wsi: fault to fake peer hangup after a delay range
Example usage, client

$ valgrind ./bin/lws-minimal-secure-streams-stress-client -c 4 --budget 40

proxy

$ valgrind ./lws-minimal-secure-streams-proxy --fault-injection "ss=mintest/wsi/timedclose(20%),ss=mintest/wsi/timedclose_ms(0..100)"
2021-07-04 10:29:54 +01:00
Andy Green
a7edd16399 logs: more internal conversion to log_cx 2021-07-01 05:20:53 +01:00
Andy Green
5d8cf03221 logs: log contexts 2021-07-01 05:20:53 +01:00
Andy Green
8e76634ed3 logs: introduce log_cx 2021-07-01 05:20:53 +01:00
Andy Green
c0680fa2b6 connect: fix cli_hostname leak breaking staggered h1 pipeline 2021-06-26 13:03:13 +01:00
Andy Green
272dba8307 http: redo redirects to follow close flow
Until now although we can follow redirects, and they can promote the
protocol from h1->h2, we couldn't handle h2 wsi reuse since there are many
states in the wsi affected by being h2.

This wipes the related states in lws_wsi_reset() and follows the generic
wsi close flow before deviating into the redirect really close to the end,
ensuring we cleaned out evidence of our previous life properly.

h2->h2 redirects work properly after this.

The max number of redirects is increased from 3 -> 4 since this was seen in
the wild with www and then geographic-based redirects.
2021-06-22 15:55:29 +01:00
Andy Green
2f9bb7a30a tls: JIT Trust
Add support for dynamically determining the CAs needed to validate server
certificates.  This allows you to avoid instantiating > 120 X.509 trusted
CA certs and have them take up heap the whole time.

Works for both openssl and mbedtls.

See READMEs/README.jit-trust.md for the documentation

You likely want the next patch for http redirect enhancements as well.
2021-06-22 15:55:29 +01:00
Andy Green
7baf400017 evlib: allow custom evlib ops
Adapt the event lib support slighly so we can pass an event lib "plugin"
header in at context creation time, and direct all event loop handling to
go via that.

This can then be lightly adapted to interface to an existing custom event
loop cleanly, without the problems of EXTERNAL_POLL.

The external loop must consult with us about the max wait timeout as shown
in the added minimal-http-server-eventlib-custom example.

The example shows a complete implementation working with a custom poll()
loop cleanly while only needing 5 ops in the custom event lib handler.
2021-06-07 08:17:49 +01:00
Catalin
373b50d7d1 tls-sessions: Use OpenSSL API to test if a session was reused 2021-04-27 15:42:53 +01:00
Andy Green
24abd699f6 http: parser: straighten out %00 legality
https://github.com/warmcat/libwebsockets/issues/2262

This adds a README explaining what can be expected if your URLs contain
%00, and adds a safe helper for urlargs-by-name that is length-based.

Contains fix for extra NUL on some headers

https://github.com/warmcat/libwebsockets/issues/2267
2021-04-10 09:21:33 +01:00
Andy Green
7d8f742594 smp: more lock assertions 2021-04-05 10:55:04 +01:00
Andy Green
51490ae6e6 Fault injection
add lws_xos: xoshiro256 PRNG
2021-04-05 10:55:04 +01:00
Catalin
d5753b6298 tls: openssl-only: client-only: session caching 2021-04-04 05:34:11 +01:00
Andy Green
3f4623bb36 lws_metrics
There are a few build options that are trying to keep and report
various statistics

 - DETAILED_LATENCY
 - SERVER_STATUS
 - WITH_STATS

remove all those and establish a generic rplacement, lws_metrics.

lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.

Openmetrics export is supported, for, eg, prometheus scraping.
2021-03-08 21:47:28 +00:00
Andy Green
95a545b741 evlib: count extant evlib handles instead of wsi 2021-01-19 17:07:33 +00:00
Andy Green
c9731c5f17 type comparisons: fixes
This is a huge patch that should be a global NOP.

For unix type platforms it enables -Wconversion to issue warnings (-> error)
for all automatic casts that seem less than ideal but are normally concealed
by the toolchain.

This is things like passing an int to a size_t argument.  Once enabled, I
went through all args on my default build (which build most things) and
tried to make the removed default cast explicit.

With that approach it neither change nor bloat the code, since it compiles
to whatever it was doing before, just with the casts made explicit... in a
few cases I changed some length args from int to size_t but largely left
the causes alone.

From now on, new code that is relying on less than ideal casting
will complain and nudge me to improve it by warnings.
2021-01-05 10:56:38 +00:00
Andy Green
93f54c61c5 ss: enforce balanced CONNECT vs DISCONNECT 2021-01-04 05:26:50 +00:00
Andy Green
0ceba15d9c lws_lifecycle
This adds some new objects and helpers for keeping and logging
info on grouped allocations, a group is, eg, SS handles or client
wsis.

Allocated objects get a context-unique "tag" string intended to replace
%p / wsi pointers etc.  Pointers quickly become confusing when
allocations are freed and reused, the tag string won't repeat
until you produce 2^64 objects in a context.

In addition the tag string documents the object group, with prefixes
like "wsi-" or "vh-" and contain object-specific additional
information like the vhost name, address / port  or the role of the wsi.
At creation time the lws code can use a format string and args
to add whatever group-specific info makes sense, eg, a wsi bound
to a secure stream can also append the guid of the secure stream,
it's copied into the new object tag and so is still available
cleanly after the stream is destroyed if the wsi outlives it.
2021-01-04 05:26:50 +00:00
Andy Green
d330dbd76a wsi: unify base wsi creation function
A few different places want to create wsis and basically repeat their
own versions of the flow.  Let's unify it into one helper in wsi.c

Also require the context lock held (this only impacts LWS_MAX_SMP > 1)
2021-01-04 05:26:50 +00:00
Andy Green
2bcae2b3b6 context: refactor destroy flow 2020-11-28 10:58:38 +00:00
Andy Green
3549a94ce6 roles: compress role ops structs
role ops are usually only sparsely filled, there are currently 20
function pointers but several roles only fill in two.  No single
role has more than 14 of the ops.  On a 32/64 bit build this part
of the ops struct takes a fixed 80 / 160 bytes then.

First reduce the type of the callback reason part from uint16_t to
uint8_t, this saves 12 bytes unconditionally.

Change to a separate function pointer array with a nybble index
array, it costs 10 bytes for the index and a pointer to the
separate array, for 32-bit the cost is

2 + (4 x ops_used)

and for 64-bit

6 + (8 x ops_used)

for 2 x ops_used it means 32-bit: 10 vs 80 / 64-bit: 22 vs 160

For a typical system with h1 (9), h2 (14), listen (2), netlink (2),
pipe (1), raw_skt (3), ws (12), == 43 ops_used out of 140, it means
the .rodata for this reduced from 32-bit: 560 -> 174 (386 byte
saving) and 64-bit: 1120 -> 350 (770 byte saving)

This doesn't account for the changed function ops calling code, two
ways were tried, a preprocessor macro and explicit functions

For an x86_64 gcc 10 build with most options, release mode,
.text + .rodata

before patch:          553282
accessor macro:        552714 (568 byte saving)
accessor functions:    553674 (392 bytes worse than without patch)

therefore we went with the macros
2020-11-28 10:58:38 +00:00
Andy Green
643a001ed8 roles: netlink
This creates a role for RFC3549 Netlink monitoring.

If the OS supports it (currently, linux) then each pt creates a wsi
with the netlink role and dumps the current routing table at pt init.
It then maintains a cache of the routing table in each pt.

Upon routing table changes an SMD message is issued as an event, and
Captive Portal Detection is triggered.

All of the pt's current connections are reassessed for routability under
the changed routing table, those that no longer have a valid route or
gateway are closed.
2020-10-19 16:35:03 +01:00
Andy Green
3fa221f9d6 smp: lws_wsi_tsi
This lets you find out the SMP Thread Service Index (tsi) that a wsi
is bound to.  This allows you to, eg, filter a global wsi list so
you can find the ones that exist in your service thread context.
2020-10-01 10:43:12 +01:00
Andy Green
6490dc3e89 smp: set tsi in fakewsi used with WAIT_CANCELLED
Together with the new lws_wsi_tsi(wsi) this lets the WAIT_CANCELLED
callback understand its tsi context, the broadcasts are done for each
pt / loop.
2020-10-01 10:41:46 +01:00
Andy Green
c6c7ab2b44 event libs: default to building as dynamically loaded plugins
Event lib support as it has been isn't scaling well, at the low level
libevent and libev headers have a namespace conflict so they can't
both be built into the same image, and at the distro level, binding
all the event libs to libwebsockets.so makes a bloaty situation for
packaging, lws will drag in all the event libs every time.

This patch implements the plan discussed here

https://github.com/warmcat/libwebsockets/issues/1980

and refactors the event lib support so they are built into isolated
plugins and bound at runtime according to what the application says
it wants to use.  The event lib plugins can be packaged individually
so that only the needed sets of support are installed (perhaps none
of them if the user code is OK with the default poll() loop).  And
dependent user code can mark the specific event loop plugin package
as required so pieces are added as needed.

The eventlib-foreign example is also refactored to build the selected
lib support isolated.

A readme is added detailing the changes and how to use them.

https://libwebsockets.org/git/libwebsockets/tree/READMEs/README.event-libs.md
2020-08-31 16:51:37 +01:00
Andy Green
76ee92afb4 coverity: 62574: wsi mux close: dumb down iterator so no dead code 2020-08-18 14:00:54 +01:00
Andy Green
1a93e73402 fakewsi: replace with smaller substructure
Currently we always reserve a fakewsi per pt so events that don't have a related actual
wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks
that look reasonable to user protocol handler code expecting a valid wsi every time.

This patch splits out stuff that user callbacks often unconditionally expect to be in
a wsi, like context pointer, vhost pointer etc into a substructure, which is composed
into struct lws at the top of it.  Internal references (struct lws is opaque, so there
are only internal references) are all updated to go via the substructre, the compiler
should make that a NOP.

Helpers are added when fakewsi is used and referenced.

If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before,
although the helpers improve consistency by zeroing down the substructure.  There is
a huge amount of user code out there over the last 10 years that did not always have
the minimal examples to follow, some of it does some unexpected things.

If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of
being able to follow the minimal examples' approach.  For PLAT_FREERTOS we don't
reserve the fakewsi in the pt any more, saving around 800 bytes.  The helpers then
create a struct lws_a (the substructure) on the stack, zero it down (but it is only
like 4 pointers) and prepare it with whatever we know like the context.

Then we cast it to a struct lws * and use it in the user protocol handler call.
In this case, the remainder of the struct lws is undefined.  However the amount of
old protocol handlers that might touch things outside of the substructure in
PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is
significant on constrained devices.

User handlers should not be touching everything in a wsi every time anyway, there
are several cases where there is no valid wsi to do the call with.  Dereference of
things outside the substructure should only happen when the callback reason shows
there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-20 06:28:52 +01:00
Andy Green
43b83c333b client: allow setting keep warm time
Until now if the generic transaction completes and the connection is idle,
there's a fixed 5s grace period to keep the connection up in case something
else wants it.

This allows setting it in the client creation info struct .keep_warm_secs.
If left at 0, then it maintains the backward-compatible 5s wait.
2020-05-06 09:06:24 +01:00
Andy Green
3d995cf7c5 fds: add sanity checks 2020-05-05 06:34:58 +01:00
Andy Green
d9f793ae12 logs: allow giving log bitfields from cmake to force build or exclusion
By default this doesn't change any existing logging behaviour at all.

But it allows you to define cmake options to force or force-disable the
build of individual log levels using new cmake option bitfields
LWS_LOGGING_BITFIELD_SET and LWS_LOGGING_BITFIELD_CLEAR.

Eg, -DLWS_LOGGING_BITFIELD_SET="(LLL_INFO)" can force INFO log level
built even in release mode.  -DLWS_LOGGING_BITFIELD_CLEAR="(LLL_NOTICE)"
will likewise remove NOTICE logging from the build regardless of
DEBUG or RELEASE mode.
2020-04-22 06:59:01 +01:00
Andy Green
a8eebc47a5 smp: helgrind and sai lock fixes 2020-04-15 20:57:56 +01:00
Andy Green
5e99b9f3ee h2: take care to retain initial sid 2020-04-06 13:22:01 +01:00
Andy Green
ee9ab7c1fc fanalyzer: workarounds and fixes
GCC -fanalyzer did find a real issue (unchecked malloc return)
but it mainly reported things that weren't true due to what
was happening outside of the particular compilation unit that
it could see.
2020-03-27 19:16:06 +00:00
Andy Green
ed837acda0 lws_set_wsi_user: allow setting to override existing internal allocation 2020-03-18 12:19:51 +00:00
Andy Green
0648a83ca2 event libs: h1 pipelining needs to manage event lib additional data during handover too 2020-03-10 19:11:15 +00:00
Andy Green
1c6735e1a0 mux: disable already optimization
This leads to problems at the moment with sticky mux.requested_POLLOUT
causing writeable to not be sent.

Remove it and always set writeable on parents for now.
2020-03-06 07:55:05 +00:00
Sakthi Kannan
9d099ba7be client: MQTT
Adds client support for MQTT QoS0 and QoS1, compatible with AWS IoT

Supports stream binding where independent client connections to the
same endpoint can mux on a single tcp + tls connection with topic
routing managed internally.
2020-03-04 12:17:49 +00:00
Andy Green
9adc64114b lws_set_wsi_user: allow setting if protocol pss size zero 2020-03-04 11:00:04 +00:00
Andy Green
ac1229f2f7 minimal-http-client-multi: add POST
This adds support for POST in both h1 and h2 queues / stream binding.

The previous queueing tried to keep the "leader" wsi who made the
actual connection around and have it act on the transaction queue
tail if it had done its own thing.

This refactors it so instead, who is the "leader" moves down the
queue and the queued guys inherit the fd, SSL * and queue from the
old leader as they take over.

This lets them operate in their own wsi identity directly and gets
rid of all the "effective wsi" checks, which was applied incompletely
and getting out of hand considering the separate lws_mux checks for
h2 and other muxed protocols alongside it.

This change also allows one wsi at a time to own the transaction for
POST.  --post is added as an option to lws-minimal-http-client-multi
and 6 extra selftests with POST on h1/h2, pipelined or not and
staggered or not are added to the CI.
2020-02-21 17:32:41 +00:00
Andy Green
774240f73b linkit: support build using public sdk
This provides support to build lws using the linkit 7697 public SDK
from here https://docs.labs.mediatek.com/resource/mt7687-mt7697/en/downloads

This toolchain has some challenges, its int32_t / uint32_t are long,
so assumptions about format strings for those being %u / %d / %x all
break.  This fixes all the cases for the features enabled by the
default cmake settings.
2020-01-17 07:45:34 +00:00
Andy Green
0bfd39135e cleaning 2020-01-05 22:17:58 +00:00
Andy Green
9cb4f25476 h2: LCCSCF_H2_MANUAL_RXFLOW and refactor txcr
This changes the approach of tx credit management to set the
initial stream tx credit window to zero.  This is the only way
with RFC7540 to gain the ability to selectively precisely rx
flow control incoming streams.

At the time the headers are sent, a WINDOW_UPDATE is sent with
the initial tx credit towards us for that specific stream.  By
default, this acts as before with a 256KB window added for both
the stream and the nwsi, and additional window management sent
as stuff is received.

It's now also possible to set a member in the client info
struct and a new option LCCSCF_H2_MANUAL_RXFLOW to precisely
manage both the initial tx credit for a specific stream and
the ongoing rate limit by meting out further tx credit
manually.

Add another minimal example http-client-h2-rxflow demonstrating how
to force a connection's peer's initial budget to transmit to us
and control it during the connection lifetime to restrict the amount
of incoming data we have to buffer.
2020-01-02 08:31:02 +00:00
Andy Green
7221bc57b5 mux children: generalize helpers out of h2 implementation
This should be a NOP for h2 support and only affects internal
apis.  But it lets us reuse the working and reliable h2 mux
arrangements directly in other protocols later, and share code
so building for h2 + new protocols can take advantage of common
mux child handling struct and code.

Break out common mux handling struct into its own type.

Convert all uses of members that used to be in wsi->h2 to wsi->mux

Audit all references to the members and break out generic helpers
for anything that is useful for other mux-capable protocols to
reuse wsi->mux related features.
2019-12-29 19:59:16 +00:00
Andy Green
1eb4d335d2 active_conns: move out dependency on ah 2019-12-23 09:25:56 +00:00
Sun Dro
10bb008dd4 lws_get_tsi
Get thread service index wsi belong to
2019-10-29 09:14:42 +00:00