1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-09 00:00:04 +01:00
Commit graph

57 commits

Author SHA1 Message Date
Andy Green
51c9e7c01e cgi: only run cleanup sul when processes pending 2021-01-17 19:20:49 +00:00
Andy Green
5af65114c9 cgi: modernize sul usage 2021-01-05 10:56:38 +00:00
Andy Green
c9731c5f17 type comparisons: fixes
This is a huge patch that should be a global NOP.

For unix type platforms it enables -Wconversion to issue warnings (-> error)
for all automatic casts that seem less than ideal but are normally concealed
by the toolchain.

This is things like passing an int to a size_t argument.  Once enabled, I
went through all args on my default build (which build most things) and
tried to make the removed default cast explicit.

With that approach it neither change nor bloat the code, since it compiles
to whatever it was doing before, just with the casts made explicit... in a
few cases I changed some length args from int to size_t but largely left
the causes alone.

From now on, new code that is relying on less than ideal casting
will complain and nudge me to improve it by warnings.
2021-01-05 10:56:38 +00:00
Andy Green
0ceba15d9c lws_lifecycle
This adds some new objects and helpers for keeping and logging
info on grouped allocations, a group is, eg, SS handles or client
wsis.

Allocated objects get a context-unique "tag" string intended to replace
%p / wsi pointers etc.  Pointers quickly become confusing when
allocations are freed and reused, the tag string won't repeat
until you produce 2^64 objects in a context.

In addition the tag string documents the object group, with prefixes
like "wsi-" or "vh-" and contain object-specific additional
information like the vhost name, address / port  or the role of the wsi.
At creation time the lws code can use a format string and args
to add whatever group-specific info makes sense, eg, a wsi bound
to a secure stream can also append the guid of the secure stream,
it's copied into the new object tag and so is still available
cleanly after the stream is destroyed if the wsi outlives it.
2021-01-04 05:26:50 +00:00
Andy Green
4343182002 logging: reduce serving logs 2020-12-01 15:38:20 +00:00
Andy Green
3549a94ce6 roles: compress role ops structs
role ops are usually only sparsely filled, there are currently 20
function pointers but several roles only fill in two.  No single
role has more than 14 of the ops.  On a 32/64 bit build this part
of the ops struct takes a fixed 80 / 160 bytes then.

First reduce the type of the callback reason part from uint16_t to
uint8_t, this saves 12 bytes unconditionally.

Change to a separate function pointer array with a nybble index
array, it costs 10 bytes for the index and a pointer to the
separate array, for 32-bit the cost is

2 + (4 x ops_used)

and for 64-bit

6 + (8 x ops_used)

for 2 x ops_used it means 32-bit: 10 vs 80 / 64-bit: 22 vs 160

For a typical system with h1 (9), h2 (14), listen (2), netlink (2),
pipe (1), raw_skt (3), ws (12), == 43 ops_used out of 140, it means
the .rodata for this reduced from 32-bit: 560 -> 174 (386 byte
saving) and 64-bit: 1120 -> 350 (770 byte saving)

This doesn't account for the changed function ops calling code, two
ways were tried, a preprocessor macro and explicit functions

For an x86_64 gcc 10 build with most options, release mode,
.text + .rodata

before patch:          553282
accessor macro:        552714 (568 byte saving)
accessor functions:    553674 (392 bytes worse than without patch)

therefore we went with the macros
2020-11-28 10:58:38 +00:00
Andy Green
22e6d5212b spawn: in discrete env setting case do it readonly
OSX changed to blow a segfault on write to .rodata, exposing that
we're dropping a NUL in what can be .rodata to set the environment
manually.  We don't do this on Linux typically because we take the
code path where execvpe() is available to do the env for us.

Adapt the code to treat it as const, and underscore it by changing
its type to be const char ** in the info struct.
2020-10-19 16:35:03 +01:00
Andy Green
44e860642b docs: switch to use main 2020-10-19 16:35:03 +01:00
Andy Green
924bd78085 clean: reduce log verbosity in various places 2020-08-10 15:04:10 +01:00
Andy Green
1a93e73402 fakewsi: replace with smaller substructure
Currently we always reserve a fakewsi per pt so events that don't have a related actual
wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks
that look reasonable to user protocol handler code expecting a valid wsi every time.

This patch splits out stuff that user callbacks often unconditionally expect to be in
a wsi, like context pointer, vhost pointer etc into a substructure, which is composed
into struct lws at the top of it.  Internal references (struct lws is opaque, so there
are only internal references) are all updated to go via the substructre, the compiler
should make that a NOP.

Helpers are added when fakewsi is used and referenced.

If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before,
although the helpers improve consistency by zeroing down the substructure.  There is
a huge amount of user code out there over the last 10 years that did not always have
the minimal examples to follow, some of it does some unexpected things.

If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of
being able to follow the minimal examples' approach.  For PLAT_FREERTOS we don't
reserve the fakewsi in the pt any more, saving around 800 bytes.  The helpers then
create a struct lws_a (the substructure) on the stack, zero it down (but it is only
like 4 pointers) and prepare it with whatever we know like the context.

Then we cast it to a struct lws * and use it in the user protocol handler call.
In this case, the remainder of the struct lws is undefined.  However the amount of
old protocol handlers that might touch things outside of the substructure in
PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is
significant on constrained devices.

User handlers should not be touching everything in a wsi every time anyway, there
are several cases where there is no valid wsi to do the call with.  Dereference of
things outside the substructure should only happen when the callback reason shows
there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-20 06:28:52 +01:00
Andy Green
da7ef0468b cgi: add spawn reap callback 2020-07-20 06:28:52 +01:00
Andy Green
286cf4357a sul: multiple timer domains
Adapt the pt sul owner list to be an array, and define two different lists,
one that acts like before and is the default for existing users, and another
that has the ability to cooperate with systemwide suspend to restrict the
interval spent suspended so that it will wake in time for the earliest
thing on this wake-suspend sul list.

Clean the api a bit and add lws_sul_cancel() that only needs the sul as the
argument.

Add a flag for client creation info to indicate that this client connection
is important enough that, eg, validity checking it to detect silently dead
connections should go on the wake-suspend sul list.  That flag is exposed in
secure streams policy so it can be added to a streamtype with
"swake_validity": true

Deprecate out the old vhost timer stuff that predates sul.  Add a flag
LWS_WITH_DEPRECATED_THINGS in cmake so users can get it back temporarily
before it will be removed in a v4.2.

Adapt all remaining in-tree users of it to use explicit suls.
2020-06-02 08:37:10 +01:00
Andy Green
d5773c01be openssl-v3: deal with deprecated SSL_CTX_load_verify_locations
Add Sai for openssl-v3 and for boringssl
2020-06-02 08:37:10 +01:00
Andy Green
b3131fdfdd cmakelist: Augean Stables refactor
Establish a new distributed CMake architecture with CMake code related to
a source directory moving to be in the subdir in its own CMakeLists.txt.
In particular, there's now one in ./lib which calls through to ones
further down the directory tree like ./lib/plat/xxx, ./lib/roles/xxx etc.

This cuts the main CMakelists.txt from 98KB -> 33KB, about a 66% reduction,
and it's much easier to maintain sub-CMakeLists.txt that are in the same
directory as the sources they manage, and conceal all the details that that
level.

Child CMakelists.txt become responsible for:

 - include_directories() definition (this is not supported by CMake
   directly, it passes it back up via PARENT_SCOPE vars in helper
   macros)

 - Addition child CMakeLists.txt inclusion, for example toplevel ->
   role -> role subdir

 - Source file addition to the build

 - Dependent library path resolution... this is now a private thing
   in the child CMakeLists.txt, it just passes back any adaptations
   to include_directories() and the LIB_LIST without filling the
   parent namespace with the details
2020-05-27 08:40:12 +01:00
Andy Green
9f1d019352 CTest: migrate and deprecate existing selftest scripts
Replace the bash selftest plumbing with CTest.

To use the selftests, build with -DLWS_WITH_MINIMAL_EXAMPLES=1
and `CTEST_OUTPUT_ON_FAILURE=1 make test` or just
`make test`.

To disable tests that require internet access, also give
-DLWS_CTEST_INTERNET_AVAILABLE=0

Remove travis and appveyor scripts on master.

Remove travis and appveyor decals on README.md.
2020-05-11 15:40:13 +01:00
Andy Green
5b9fe01863 build: release mode compile fixes 2020-03-10 06:45:24 +00:00
Andy Green
9a1f184915 rtos diet: http: remove headers at buildtime according to config
Headers related to ws or h2 are now elided if the ws or h2 role
is not enabled for build.  In addition, a new build-time option
LWS_WITH_HTTP_UNCOMMON_HEADERS on by default allows removal of
less-common http headers to shrink the parser footprint.

Minilex is adapted to produce 8 different versions of the lex
table, chosen at build-time according to which headers are
included in the build.

If you don't need the unusual headers, or aren't using h2 or ws,
this chops down the size of the ah and the rodata needed to hold
the parsing table from 87 strings / pointers to 49, and the
parsing table from 1177 to 696 bytes.
2020-03-04 11:00:04 +00:00
Andy Green
8a7e0edb7d lws_spawn_piped: break out from cgi
The vfork optimized spawn, stdxxx and terminal handling in the cgi
implementation is quite mature and sophisticated, and useful for
other things unrelated to cgi.  Break it out into its own public
api under LWS_WITH_SPAWN, off by default.

Expand it so the parent wsi is optional, and the role and protocol
bindings for stdxxx pipes can be set.  Allow optional sul timeout
and external lws_dll2 owner for extant children.

Remove inline style from minimal http-server-cgi
2020-02-21 17:32:41 +00:00
Andy Green
b0b8a684ed event-ops: rename accept member to sock_accept
Some toolchains have lwip accept() as a preprocessor define...
2020-01-15 12:20:50 +00:00
Andy Green
6f2230a993 role structs to const
Indicate these are immutable (they're already treated as
such) and can go in .rodata
2020-01-15 06:31:19 +00:00
Andy Green
0bfd39135e cleaning 2020-01-05 22:17:58 +00:00
Andy Green
7221bc57b5 mux children: generalize helpers out of h2 implementation
This should be a NOP for h2 support and only affects internal
apis.  But it lets us reuse the working and reliable h2 mux
arrangements directly in other protocols later, and share code
so building for h2 + new protocols can take advantage of common
mux child handling struct and code.

Break out common mux handling struct into its own type.

Convert all uses of members that used to be in wsi->h2 to wsi->mux

Audit all references to the members and break out generic helpers
for anything that is useful for other mux-capable protocols to
reuse wsi->mux related features.
2019-12-29 19:59:16 +00:00
Andy Green
c4ab815aaf _GNU_SOURCE: only define if not already defined
https://github.com/warmcat/libwebsockets/issues/1803
2019-12-22 18:17:45 +00:00
Andy Green
092ebf9879 cgi: minimal example 2019-11-20 05:20:59 +00:00
Andy Green
1968edcf44 cgi: reenable h1 with chunking 2019-11-16 09:00:15 +00:00
Zhiwen Zheng
70d2ca94b5 cgi: fix passing cgi envs using setenv() 2019-10-12 12:41:14 +01:00
Andy Green
f9f6bb66fe lws_validity: unified connection validity tracking
Refactor everything around ping / pong handling in ws and h2, so there
is instead a protocol-independent validity lws_sul tracking how long it
has been since the last exchange that confirms the operation of the
network connection in both directions.

Clean out periodic role callback and replace the last two role users
with discrete lws_sul for each pt.
2019-09-22 09:35:07 -07:00
Andy Green
d7f0521aeb private.h: rename to contain dir
Having unique private header names is a requirement of a particular
platform build system it's desirable to work with
2019-08-15 10:49:52 +01:00
Andy Green
26319663f7 license: switch LGPLv2.1+SLE parts to MIT 2019-08-14 10:44:38 +01:00
Andy Green
fc5defdd2a COVA10299: check lws_change_pollfd 2019-07-13 13:39:50 -07:00
Andy Green
d727c89d79 cmake: miniz: add as option 2019-07-03 19:46:23 +01:00
Andy Green
4692c1a7ee rtos: clean warnings where uint32_t is an unsigned long 2019-06-07 11:11:46 +01:00
Andy Green
af817c6532 cgi: fix h2 timeouts 2019-04-06 05:52:23 +08:00
Andy Green
f74326e6e1 cgi: fix stdin cgiwsi leak when closed early 2019-03-10 08:02:02 +08:00
Andy Green
9123ca6bef cgi: fix stdout close to http close
On h1, cgi stdout close doesn't prompt the http close, instead it
times out.  Fix that so we also close on h1, and make the close
action itself on http timeout less drastic.

As it was, GnuTLS actually marks the close as a fatal TLS error.
2019-01-29 12:25:20 +08:00
Andy Green
08b5ad9299 role: raw-proxy 2018-12-01 11:05:59 +08:00
Andy Green
9bed6d6fc6 clean: general whitespace cleanup 2018-11-23 08:47:56 +08:00
Andy Green
d702b83d10 uv: allocate watcher
Until now the uv watcher has been composed in the wsi.

This works fine except in the case of a client wsi that
meets a redirect when the event loop is libuv with its
requirement for handle close via the event loop.

We want to reuse the wsi, since the originator of it has
a copy of the wsi pointer, and we want to conceal the
redirect.  Since the redirect is commonly to a different
IP, we want to keep the wsi alive while closing its
socket cleanly.  That's not too difficult, unless you are
using uv.

With UV the comoposed watcher is a disaster, since after
the close is requested the wsi will start to reconnect.
We tried to deal with that by copying the uv handle and
freeing it when the handle close finalizes.  But it turns
out the handle is in a linked-list scheme in uv.

This patch hopefully finally solves it by giving the uv
handle its own allocation from the start.  When we want
to close the socket and reuse the wsi, we simply take
responsibility for freeing the handle and set the wsi
watcher pointer to NULL.
2018-10-13 12:43:13 +08:00
Andy Green
d03c57b87f quench logging 2018-10-13 08:16:27 +08:00
Andy Green
5c0b0450f2 client: bind and drop protocol like server
HTTP server protocols have had for a while LWS_CALLBACK_HTTP_DROP/BIND_PROTOCOL
callbacks that mark when a wsi is attched to a protocol and detached.

It turns out this is generally useful for everything to know when a wsi is
joining a protocol and definitively completely finished with a protocol.

Particularly with client wsi where you provided the userdata externally, this
makes a clear point to free() it on the protocol binding being dropped.

This patch adds protocol bind / unbind callbacks to the role definition and
lets them operate on all roles.  For the various roles

HTTP server: LWS_CALLBACK_HTTP_BIND/DROP_PROTOCOL as before
HTTP client: LWS_CALLBACK_CLIENT_HTTP_BIND/DROP_PROTOCOL
ws server:   LWS_CALLBACK_WS_SERVER_BIND/DROP_PROTOCOL
ws client:   LWS_CALLBACK_WS_CLIENT_BIND/DROP_PROTOCOL
raw file:    LWS_CALLBACK_RAW_FILE_BIND/DROP_PROTOCOL
raw skt:     LWS_CALLBACK_RAW_SKT_BIND/DROP_PROTOCOL
2018-08-18 14:11:29 +08:00
Andy Green
d461f46a97 libwebsockets.h: clean out some boilerplate better put in core/private.h
https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
Andy Green
69d9cf2e6b coverity: cleanup 2018-08-14 08:00:30 +08:00
Andy Green
f2f96857d6 fd_cloexec: add and use lws_open wrapper and lws_plat_apply_FD_CLOEXEC() on cgi 2018-06-23 12:56:21 +08:00
Andy Green
ac3bd36c60 vhost_destroy: use vhost wsi reference counting to trigger destroy
This changes the vhost destroy flow to only hand off the listen
socket if another vhost sharing it, and mark the vhost as
being_destroyed.

Each tsi calls lws_check_deferred_free() once a second, if it sees
any vhost being_destroyed there, it closes all wsi on its tsi on
the same vhost, one time.

As the wsi on the vhost complete close (ie, after libuv async close
if on libuv event loop), they decrement a reference count for all
wsi open on the vhost.  The tsi who closes the last one then
completes the destroy flow for the vhost itself... it's random
which tsi completes the vhost destroy but since there are no
wsi left on the vhost, and it holds the context lock, nothing
can conflict.

The advantage of this is that owning tsi do the close for wsi
that are bound to the vhost under destruction, at a time when
they are guaranteed to be idle for service, and they do it with
both vhost and context locks owned, so no other service thread
can conflict for stuff protected by those either.

For the situation the user code may have allocations attached to
the vhost, this adds args to lws_vhost_destroy() to allow destroying
the user allocations just before the vhost is freed.
2018-06-18 09:11:46 +08:00
Andy Green
d2bdb60a17 cgi: fix for https git server 2018-06-16 09:35:07 +08:00
Andy Green
a177285b9c cgi: fix QUERY_STRING 2018-05-18 08:40:18 +08:00
Andy Green
502130d999 refactor: split out adoption and client apis to core
- split raw role into separate skt and file

 - remove all special knowledge from the adoption
   apis and migrate to core

 - remove all special knowledge from client_connect
   stuff, and have it discovered by iterating the
   role callbacks to let those choose how to bind;
   migrate to core

 - retire the old deprecated client apis pre-
   client_connect_info
2018-05-11 10:29:08 +08:00
Andy Green
c4dc102a0b windows: cleanup wrong and duplicated socket validity helpers
https://github.com/warmcat/libwebsockets/issues/1259
2018-05-06 07:22:25 +08:00
Andy Green
de064fd65a refactor: core code in lib/core and private-libwebsockets.h to core/private.h
This commit is coverity-clean as tested

cmake .. -DLWS_WITH_MINIMAL_EXAMPLES=1 -DLWS_WITHOUT_EXTENSIONS=1 -DLWS_WITH_ACME=1 -DLWS_WITH_LWSWS=1 -DLWS_WITH_LIBUV=1 -DLWS_WITH_HTTP2=1 -DLWS_WITHOUT_CLIENT=0 -DLWS_WITHOUT_SERVER=0 -DLWS_UNIX_SOCK=1 -DLWS_WITH_TLS=0 -DLWS_WITH_MBEDTLS=0 -DLWS_WITH_CGI=1 -DCMAKE_BUILD_TYPE=DEBUG -DLWS_WITH_GENERIC_SESSIONS=1 -DLWS_WITH_RANGES=1 -DLWS_ROLE_WS=1 -DLWS_MAX_SMP=16 -DLWS_ROLE_H1=1 -DLWS_WITH_WOLFSSL=0 -DLWS_WITH_LIBEV=0 -DLWS_WITH_LIBEVENT=1
2018-05-03 10:49:36 +08:00
Andy Green
d37b383edc refactor: apply ops structs to event loop handlers 2018-04-29 10:44:36 +08:00