There are a few build options that are trying to keep and report
various statistics
- DETAILED_LATENCY
- SERVER_STATUS
- WITH_STATS
remove all those and establish a generic rplacement, lws_metrics.
lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.
Openmetrics export is supported, for, eg, prometheus scraping.
This is a huge patch that should be a global NOP.
For unix type platforms it enables -Wconversion to issue warnings (-> error)
for all automatic casts that seem less than ideal but are normally concealed
by the toolchain.
This is things like passing an int to a size_t argument. Once enabled, I
went through all args on my default build (which build most things) and
tried to make the removed default cast explicit.
With that approach it neither change nor bloat the code, since it compiles
to whatever it was doing before, just with the casts made explicit... in a
few cases I changed some length args from int to size_t but largely left
the causes alone.
From now on, new code that is relying on less than ideal casting
will complain and nudge me to improve it by warnings.
Adapt the pt sul owner list to be an array, and define two different lists,
one that acts like before and is the default for existing users, and another
that has the ability to cooperate with systemwide suspend to restrict the
interval spent suspended so that it will wake in time for the earliest
thing on this wake-suspend sul list.
Clean the api a bit and add lws_sul_cancel() that only needs the sul as the
argument.
Add a flag for client creation info to indicate that this client connection
is important enough that, eg, validity checking it to detect silently dead
connections should go on the wake-suspend sul list. That flag is exposed in
secure streams policy so it can be added to a streamtype with
"swake_validity": true
Deprecate out the old vhost timer stuff that predates sul. Add a flag
LWS_WITH_DEPRECATED_THINGS in cmake so users can get it back temporarily
before it will be removed in a v4.2.
Adapt all remaining in-tree users of it to use explicit suls.
https://libwebsockets.org/pipermail/libwebsockets/2019-April/007937.html
thanks to Bruce Perens for noting it.
This doesn't change the intention or status of the CC0 files, they were
pure CC0 before (ie, public domain) and they are pure CC0 now. It just
gets rid of the (C) part at the top of the dedication which may be read
to be a bit contradictory since the purpose is to make it public domain.
This has no effect on user code or backward compatibility.
It moves the in-tree public api header libwebsockets.h from ./lib
to ./include, and introduces a dir ./include/libwebsockets/
The single public api header is split out into 31 sub-headers
in ./include/libwebsockets. ./include/libwebsockets.h contains
some core types and platform adaptation code, but the rest of it
is now 31 #include <libwebsockets/...>
At install time, /usr/[local/]include/libwebsockets.h is installed
as before, along now with the 31 sub-headers in ...include/libwebsockets/
There's no net effect on user code.
But the api header is now much easier to maintain and study, with 31
topic-based sub headers.
This adds a plugin that interfaces to libjsongit2
https://warmcat.com/git/libjsongit2
to provide a per-vhost service for presenting bare git repos in a
web interface.
For h1 / ws, a combination of removing POLLIN wait and
stashing any unused rx lets us immediately respond to
rx flow control requests in a simple and effective way,
because the tcp socket is the stream.
But for muxed protocols like h2, that technique cannot
be used because we cannot silence the whole bundle of
streams because one can't handle any more rx dynamically.
There are control frames and content for other streams
serialized inbetween the flow controlled stream content.
We have no choice but to read to so we can see the other
things. Therefore for muxed protocols like h2 and spdy,
rx flow control boils down to tx credit manipulation
on individual streams to staunch the flow at the peer.
However that requires a round trip to take effect, any
transmitted packets that were in flight before the tx credit
reduction arrives at the remote peer are still going to come
and have to be dealt with by adding them to the stash.
This patch introduces lws_buflist scatter-gather type
buffer management for rxflow handling, so we can append
buffer segments in a linked-list to handle whatever rx
is unavoidably in flight on a stream that is trying to
assert rx flow control.
This is just an internal mass change of LWS_NO_EXTENSIONS to
LWS_WITHOUT_EXTENSIONS to match the public name and eliminate
all instances of LWS_NO_EXTENSIONS.
This enables selected things from -Wextra, can't use -Wextra because it is
fussy enough to complain about unused params on functions... they are
there for a reason.
-Wsign-compare
-Wignored-qualifiers
not -Wimplicit-fallthrough=3 ... only on gcc 7
-Wtype-limits
-Wuninitialized
not -Wclobbered ... only on gcc 7ish
fix the warnings everywhere they were found.
HTTP/2 support is now able to serve the test server, complete with
websockets, from a single vhost.
- This works the same with both OpenSSL and mbedTLS.
- POST is now wired up and works (also for file upload).
- CGI is wired up and works.
- Redirect is adapted and works
- lwsws works.
- URI urldecode, sanitation and argument parsing wired up for :path
valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally)
h2spec reports:
$ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1
...
145 tests, 145 passed, 0 skipped, 0 failed"
Incorporates:
- "https://github.com/warmcat/libwebsockets/pull/1039
Fixes issue with -Werror=unused-variable flag
- 2c843a1395
ssl: fix infinite loop on client cert verification failure
Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>"
Caused and fixes Coverity 184887 - 184892
For some targets like ESP32, the ah pool is mainly sitting idle wasting memory.
For HTTP/2, if the client sends a series of pipelined headers on different SIDs
that exist simultaneously, there is no way to stall the headers to wait for an
ah, because we must read the stream for stuff like WINDOW_UPDATE on the other
streams.
In both these cases having the ability to free unused ah completely and allocate
more dynamically if there is memory is useful, so this patch makes the ah pool
an initially-empty linked list that allocates on demand up to the "max pool
size" limit from the context info. When nobody wants an ah, it is freed (if
someone was waiting for it, it is directly reused).
For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc
according to how many streams the client opens, which can be controlled by
SETTINGS.
Adds a new api lws_vhost_destroy(struct lws_vhost *) which allows dynamic removal of vhosts.
The external api calls two parts of internal helpers that get reused for context destroy.
The second part is called deferred by 5s... this is to ensure that event library objects
composed into structs owned by the vhost all have a chance to complete their close
asynchronously. That should happen immediately, but it requires us to return to the
event loop first.
The vhost being removed is deleted from the context vhost list by the first part, and does
not block further removals or creation during the delay for the deferred freeing of the
vhost memory.
Part 1:
- if the vhost owned a listen socket needed by other vhosts listening on same iface + port, the listen
socket is first handed off to another vhost so it stays alive
- all wsi still open on the vhost are forcibly closed (including any listen socket still attached)
- inform all active protocols on the vhost they should destroy themselves
- remove vhost from context vhost list (can no longer be found by incoming connections)
- add to a "being destroyed" context list and schedule the second part to be called in 5s
Part 2:
- remove us from the being destroyed list
- free all allocations owned by the vhost
- zero down the vhost and free the vhost itself
In libwebsockets-test-server, you can send it a SIGUSR1 to have it toggle the creation and destruction of
a second vhost on port + 1.
1) This makes lwsws run a parent process with the original permissions.
But this process is only able to respond to SIGHUP, it doesn't do anything
else.
2) You can send this parent process a SIGHUP now to cause it to
- close listening sockets in existing lwsws processes
- mark those processes as to exit when the number of active connections
on the falls to zero
- spawn a fresh child process from scratch, using latest configuration
file content, latest plugins, etc. It can now reopen listening sockets
if it chooses to, or open different listen ports or whatever.
Notes:
1) lws_context_destroy() has been split into two pieces... the reason for
the split is the first part closes the per-vhost protocols, but since
they may have created libuv objects in the per-vhost protocol storage,
these cannot be freed until after the loop has been run.
That's the purpose of the second part of the context destruction,
lws_context_destroy2().
For compatibility, if you are not using libuv, the first part calls the
second part. However if you are using libuv, you must now call the
second part from your own main.c after the first part.
This adds a context creation-time member that points to something
that should be freed when the context is destroyed.
It's in preparation for context deprecation, when a context might
be destroyed asynchronously... a related external with the
lifetime of thee context should also be freed at that time.
Adapt lwsws to use it with the context "strings" (also used for
aligned structs created by the config) allocation.
Actually lwsws doesn't need his own protocol handler even for http
any more. The default http handler in lws should do everything.
Move the cgi routing into lws default http protocol handler, and
delete lwsws one. Remove all protocols from lwsws so the lws
default one gets used.
With this, and the earlier move of lejp into lws, lwsws itself
becomes 15.5KB of x86_64 (mainly conf parsing).
Signed-off-by: Andy Green <andy@warmcat.com>
If OOT lws plugins will be packaged as separate projects,
they're going to want to install their plugins somewhere
that makes sense for the package instead of one big lws
plugin dir.
This patch changes info to have a const char ** to a NULL
terminated array of directories it should search for
plugins. lwsws knows about this and you can add to the
dir array using config fragments like
{
"global": {
"plugin-dir": "/usr/local/share/coherent-timeline/plugins"
}
}
if the config fragment in /etc/lwsws/conf.d/ is also managed by the
package with the plugin, it can very cleanly add and remove itself
from lwsws based on package install status.
Signed-off-by: Andy Green <andy@warmcat.com>
https://github.com/warmcat/libwebsockets/issues/489
This
1) fixes the vhost changes on master
2) works around the ah pool changes
3) fixes some other build problems that appeared
4) hacks out physical flow control for internal streams
5) updates the advertised protocol to h2 needed by, eg, chrome 51
That gets it able to serve small (<4K, ie, one packet) files over http2
Signed-off-by: Andy Green <andy@warmcat.com>
This adds support for dynamically loaded plugins at runtime, which
can expose their own protocols or extensions transparently.
With these changes lwsws defaults to OFF in cmake, and if enabled it
automatically enables plugins and libuv support.
Signed-off-by: Andy Green <andy@warmcat.com>
This makes a start on the LibWebSockets WebServer.
The app cmake build support and JSON config parsing are implemented and
the app can start, create the vhosts, listen and serve file:// mounts on
them.
Signed-off-by: Andy Green <andy@warmcat.com>
