1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-16 00:00:07 +01:00
Commit graph

90 commits

Author SHA1 Message Date
Andy Green
3f4623bb36 lws_metrics
There are a few build options that are trying to keep and report
various statistics

 - DETAILED_LATENCY
 - SERVER_STATUS
 - WITH_STATS

remove all those and establish a generic rplacement, lws_metrics.

lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.

Openmetrics export is supported, for, eg, prometheus scraping.
2021-03-08 21:47:28 +00:00
Andy Green
b6aa4c2d5d solaris: pthread is not a ptr 2021-03-08 20:44:10 +00:00
Andy Green
09b9ac6e33 minimal-http-client: set option to fix broken server window update 2021-02-28 12:10:26 +00:00
Andy Green
cb5c0059a3 sai-resource 2021-02-24 08:56:27 +00:00
Andy Green
3fe08ce5d8 fault injection 2021-02-20 13:54:35 +00:00
Andy Green
8d48d8f3fb ctest: mbedtls: disable spawned server valgrind
When the number of parallel mbedtls ctest runs exceeds the number of free cpus in the
builder, we get too slow when under valgrind to pass the client tests before they
timeout.
2021-02-03 08:49:59 +00:00
Andy Green
6497220e87 plugins: add LWS_BUILD_HASH to header and require match
Also prioritize LD_LIBRARY_PATH check for plugins first

Iterate through paths in LD_LIBRARY_PATH in order

Warn on failed plugins init but continue protocol init
2021-01-28 05:52:41 +00:00
Andy Green
56f7007185 ctest: adapt background script 2021-01-26 07:48:10 +00:00
Andy Green
d5c90645c4 ctest: valgrind on hcm 2021-01-26 07:47:54 +00:00
Andy Green
06662a64fa LE-root-cert-update
warmcat.com and libwebsockets.org use Let's Encrypt certificates... LE
have changed their CA signing arrangements and after 2021-01-12 (the
point I renewed the LE server certs and received one signed using the
new arrangements) it's required to trust new root certs for the examples
to connect to warmcat.com and libwebsockets.org.

https://letsencrypt.org/2020/09/17/new-root-and-intermediates.html

This updates the in-tree CA copies, the remote policies on warmcat.com
have also been updated.

Just goes to show for real client infrastructure, you need to run your own
CA (that doesn't have to be trusted by anything outside the clients)
where you can control the CA lifetime.
2021-01-13 04:32:13 +00:00
Andy Green
c9731c5f17 type comparisons: fixes
This is a huge patch that should be a global NOP.

For unix type platforms it enables -Wconversion to issue warnings (-> error)
for all automatic casts that seem less than ideal but are normally concealed
by the toolchain.

This is things like passing an int to a size_t argument.  Once enabled, I
went through all args on my default build (which build most things) and
tried to make the removed default cast explicit.

With that approach it neither change nor bloat the code, since it compiles
to whatever it was doing before, just with the casts made explicit... in a
few cases I changed some length args from int to size_t but largely left
the causes alone.

From now on, new code that is relying on less than ideal casting
will complain and nudge me to improve it by warnings.
2021-01-05 10:56:38 +00:00
Andy Green
0ceba15d9c lws_lifecycle
This adds some new objects and helpers for keeping and logging
info on grouped allocations, a group is, eg, SS handles or client
wsis.

Allocated objects get a context-unique "tag" string intended to replace
%p / wsi pointers etc.  Pointers quickly become confusing when
allocations are freed and reused, the tag string won't repeat
until you produce 2^64 objects in a context.

In addition the tag string documents the object group, with prefixes
like "wsi-" or "vh-" and contain object-specific additional
information like the vhost name, address / port  or the role of the wsi.
At creation time the lws code can use a format string and args
to add whatever group-specific info makes sense, eg, a wsi bound
to a secure stream can also append the guid of the secure stream,
it's copied into the new object tag and so is still available
cleanly after the stream is destroyed if the wsi outlives it.
2021-01-04 05:26:50 +00:00
Andy Green
eb5f437578 h2: post: add states to wait for body 2020-12-24 16:14:36 +00:00
Andy Green
49b301b53b minimal: http-client-multi: set h2 or h1 via alpn 2020-12-24 16:14:36 +00:00
Andy Green
1d8be99cff cmake: latest cmake shows dep warnings for scripts < 2.8.12 2020-12-06 19:44:54 +00:00
Andy Green
050d23f0be sai: mac m1 2020-12-06 19:06:07 +00:00
Andy Green
1ebe27e38f cmake: mark all the subprojects as explicitly C
We don't need a C++ compiler but if we don't spell it out, cmake
assumes it C + C++ compiler needed.
2020-08-31 16:51:36 +01:00
Andy Green
629c8138b1 sai: freebsd 2020-08-10 15:04:10 +01:00
Andy Green
4cd381f933 cmake: tls: wolfssl
PARENT_SCOPE needs adjusting in a few places for wolfssl to work, and
we need a second level export of USE_WOLFSSL through lib/CMakeLists.txt

Add noi/f32 Sai build for WOLFSSL + MINIMAL_EXAMPLES
2020-08-10 15:04:10 +01:00
Andy Green
30f3606b0e context: option to disable system state management 2020-06-27 07:57:22 +01:00
Andy Green
34de22b648 cmake: cleanups 2020-06-18 08:53:09 +01:00
Andy Green
b324f0e43b cmake: provide LIBWEBSOCKETS_DEP_LIBS in CONFIG 2020-06-16 19:45:35 +01:00
Andy Green
286cf4357a sul: multiple timer domains
Adapt the pt sul owner list to be an array, and define two different lists,
one that acts like before and is the default for existing users, and another
that has the ability to cooperate with systemwide suspend to restrict the
interval spent suspended so that it will wake in time for the earliest
thing on this wake-suspend sul list.

Clean the api a bit and add lws_sul_cancel() that only needs the sul as the
argument.

Add a flag for client creation info to indicate that this client connection
is important enough that, eg, validity checking it to detect silently dead
connections should go on the wake-suspend sul list.  That flag is exposed in
secure streams policy so it can be added to a streamtype with
"swake_validity": true

Deprecate out the old vhost timer stuff that predates sul.  Add a flag
LWS_WITH_DEPRECATED_THINGS in cmake so users can get it back temporarily
before it will be removed in a v4.2.

Adapt all remaining in-tree users of it to use explicit suls.
2020-06-02 08:37:10 +01:00
Andy Green
b3131fdfdd cmakelist: Augean Stables refactor
Establish a new distributed CMake architecture with CMake code related to
a source directory moving to be in the subdir in its own CMakeLists.txt.
In particular, there's now one in ./lib which calls through to ones
further down the directory tree like ./lib/plat/xxx, ./lib/roles/xxx etc.

This cuts the main CMakelists.txt from 98KB -> 33KB, about a 66% reduction,
and it's much easier to maintain sub-CMakeLists.txt that are in the same
directory as the sources they manage, and conceal all the details that that
level.

Child CMakelists.txt become responsible for:

 - include_directories() definition (this is not supported by CMake
   directly, it passes it back up via PARENT_SCOPE vars in helper
   macros)

 - Addition child CMakeLists.txt inclusion, for example toplevel ->
   role -> role subdir

 - Source file addition to the build

 - Dependent library path resolution... this is now a private thing
   in the child CMakeLists.txt, it just passes back any adaptations
   to include_directories() and the LIB_LIST without filling the
   parent namespace with the details
2020-05-27 08:40:12 +01:00
Andy Green
36ec353dac client: allow setting overall connect timeout from vh init
Add a member to the vh init struct allowing control of the overall
connection wait introduced in an earlier patch.  Set it to 20s
by default.


The timeout_secs member controls the individual DNS result
connect timeout and is reduced to 5s by default.
2020-05-26 17:05:39 +01:00
Andy Green
9f1d019352 CTest: migrate and deprecate existing selftest scripts
Replace the bash selftest plumbing with CTest.

To use the selftests, build with -DLWS_WITH_MINIMAL_EXAMPLES=1
and `CTEST_OUTPUT_ON_FAILURE=1 make test` or just
`make test`.

To disable tests that require internet access, also give
-DLWS_CTEST_INTERNET_AVAILABLE=0

Remove travis and appveyor scripts on master.

Remove travis and appveyor decals on README.md.
2020-05-11 15:40:13 +01:00
Kyle Greenwell
bebd3f71a6 h2: add prior knowledge support 2020-05-08 18:56:17 +01:00
Andy Green
eca75ee14d http: pipelining: fixes from xenial sai 2020-04-15 20:35:13 +01:00
Andy Green
d3308df40f windows: pthreads
Add support for external pthreads lib on windows and some docs about how to do.

It can build with LWS_WITH_THREADPOOL and LWS_WITH_MINIMAL_EXAMPLES including the
pthreads-dependent ones without warnings or errors on windows platform as well with this.

pthreads_t can be anything, including a struct - not a pointer-to-a-struct
but the struct itself.  These can't be cast to a void * for printing as they can
on linux, where the base type is a pointer.

Let's fix all the usage of those to determine their own thread index in terms
of the meaning to the program rather than as a tid.
2020-04-07 14:05:21 +01:00
Andy Green
954fcf2bc3 windows: clean LWS_WITH_MINIMAL_EXAMPLES warnings
Fix pthreads detection in the minimal examples and add it where needed.
Fix unistd.h include to be conditional on not WIN32

With this, -DLWS_WITH_MINIMAL_EXAMPLES=1 is happy and warning-free
on windows.
2020-04-07 10:09:51 +01:00
Andy Green
a60cb84c9e captive portal
Implement Captive Portal detection support in lws, with the actual
detection happening in platform code hooked up by lws_system_ops_t.

Add an implementation using Secure Streams as well, if the policy
defines captive_portal_detect streamtype, a SS using that streamtype
is used to probe if it's behind a captive portal.
2020-03-11 12:44:01 +00:00
Andy Green
eaab6e28b0 client: unify post tls accept handling 2020-03-07 20:03:58 +00:00
Andy Green
28ce32af64 client: secure streams
Secure Streams is an optional layer on top of lws that separates policy
like endpoint selection and tls cert validation into a device JSON
policy document.

Code that wants to open a client connection just specifies a streamtype name,
and no longer deals with details like the endpoint, the protocol (!) or anything
else other than payloads and optionally generic metadata; the JSON policy
contains all the details for each streamtype.  h1, h2, ws and mqtt client
connections are supported.

Logical secure streams outlive any particular connection and supports "nailed-up"
connectivity regardless of underlying connection stability.
2020-03-04 12:17:49 +00:00
Andy Green
d88d41090e cleaning: windows 2020-03-04 12:17:49 +00:00
Andy Green
bb1a8d37fb rtos diet: make basic auth support a config option on by default
Remove wrapper content for < tls v1.2
2020-03-04 11:00:04 +00:00
Andy Green
0dd23d7073 tls restriction: account for pipelining and add test 2020-03-01 08:14:43 +00:00
Andy Green
af2f1f9a87 event-lib-glib
Adds support for glib event library.
2020-02-21 17:32:41 +00:00
Andy Green
ac1229f2f7 minimal-http-client-multi: add POST
This adds support for POST in both h1 and h2 queues / stream binding.

The previous queueing tried to keep the "leader" wsi who made the
actual connection around and have it act on the transaction queue
tail if it had done its own thing.

This refactors it so instead, who is the "leader" moves down the
queue and the queued guys inherit the fd, SSL * and queue from the
old leader as they take over.

This lets them operate in their own wsi identity directly and gets
rid of all the "effective wsi" checks, which was applied incompletely
and getting out of hand considering the separate lws_mux checks for
h2 and other muxed protocols alongside it.

This change also allows one wsi at a time to own the transaction for
POST.  --post is added as an option to lws-minimal-http-client-multi
and 6 extra selftests with POST on h1/h2, pipelined or not and
staggered or not are added to the CI.
2020-02-21 17:32:41 +00:00
Andy Green
271ca836c8 event lib: update http client multi to work with it and clean destroy flow
Add selectable event lib support to minimal-http-client-multi and
clean up context destroy flow so we can use lws_destroy_context() from
inside the callback to indicate we want to end the event loop, without
using the traditional "interrupted" flag and in a way that works no
matter which event loop backend is being used.
2020-02-04 14:16:18 +00:00
Andy Green
0bfd39135e cleaning 2020-01-05 22:17:58 +00:00
Andy Green
d8ccfc2370 lws_system: helpers for attaching to existing event loop from other threads
In the case code is composed into a single process, but it isn't monolithic in the
sense it's made up of modular "applications" that are written separate projects,
provide a way for the "applications" to request a callback from the lws event loop
thread context safely.

From the callback the applications can set up their operations on the lws event
loop and drop their own thread.

Since it requires system-specific locking to be threadsafe, provide a non-threadsafe
helper and then indirect the actual usage through a user-defined lws_system ops
function pointer that wraps the unsafe api with the system locking to make it safe.
2020-01-05 22:17:58 +00:00
Andy Green
9cb4f25476 h2: LCCSCF_H2_MANUAL_RXFLOW and refactor txcr
This changes the approach of tx credit management to set the
initial stream tx credit window to zero.  This is the only way
with RFC7540 to gain the ability to selectively precisely rx
flow control incoming streams.

At the time the headers are sent, a WINDOW_UPDATE is sent with
the initial tx credit towards us for that specific stream.  By
default, this acts as before with a 256KB window added for both
the stream and the nwsi, and additional window management sent
as stuff is received.

It's now also possible to set a member in the client info
struct and a new option LCCSCF_H2_MANUAL_RXFLOW to precisely
manage both the initial tx credit for a specific stream and
the ongoing rate limit by meting out further tx credit
manually.

Add another minimal example http-client-h2-rxflow demonstrating how
to force a connection's peer's initial budget to transmit to us
and control it during the connection lifetime to restrict the amount
of incoming data we have to buffer.
2020-01-02 08:31:02 +00:00
Andy Green
36de0ada7d http client basic auth add helper and example 2019-12-22 18:17:45 +00:00
Andy Green
fb25b64b83 minimal http client multi: rename struct user so it cant conflict 2019-12-09 14:48:54 +00:00
Andy Green
efbab4c602 minimal client: add --path option 2019-12-03 07:33:19 +00:00
Andy Green
0f7f27801e http redirect: 303: force method to GET
This teaches http client stuff how to handle 303 redirects... these
can happen after POST where the server side wants you to come back with
a GET to the Location: mentioned.

lws client will follow the redirect and force GET, this works for both
h1 and h2.  Client protocol handler has to act differently if it finds
it is connecting for the initial POST or the subsequent GET, it can
find out which by checking a new api lws_http_is_redirected_to_get(wsi)
which returns nonzero if in GET mode.

Minimal example for server form-post has a new --303 switch to enable
this behaviour there and the client post example has additions to
check lws_http_is_redirected_to_get().
2019-11-17 10:47:01 +00:00
Andy Green
0ab4a707fb format strings: ban %.*s as some platforms lack it
The %.*s is very handy to print strings where you have a length, but
there is no NUL termination.  It's quite widely supported but at least
one vendor RTOS toolchain doesn't have it.

Since there aren't that many uses of it yet, audit all uses and
convert to a new helper lws_strnncpy() which uses the smaller of
two lengths.
2019-11-04 13:49:14 +00:00
Andy Green
6a56855284 minimal client http: add lws_get_peer_simple example 2019-11-02 10:11:20 +00:00
Andy Green
127e53cf98 client: multipart mime generation helpers
lws has been able to generate client multipart mime as shown
in minimal-http-client-post, but it requires a lot of user
boilerplate to handle the boundary, related transaction header,
and multipart headers.

This patch adds a client creation flag to indicate it will
carry multipart mime, which autocreates the boundary string
and applies the transaction header with it, and an api to
form the boundary headers between the different mime parts
and the terminating boundary.
2019-10-12 12:41:14 +01:00
Andy Green
a83c6439e6 client h2: auth bearer option flag
This affects max header size since we use the latter half
of the pt_serv_buf to prepare the (possibly huge) auth token.

Adapt the pt_serv_buf_size in the hugeurl example.
2019-10-12 12:41:14 +01:00