1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-16 00:00:07 +01:00
Commit graph

71 commits

Author SHA1 Message Date
Andy Green
72b245ea38 h2: fix breakage with LWS_WITH_HTTP2=0 2020-08-31 16:51:37 +01:00
Andy Green
146858fb54 coverity: 231926: clear false positive by showing coverity what it wants to see
We can't get here without testing for COLON_PATH existing in http2.c as part of
the h2spec pass code.

		if (!lws_hdr_total_length(h2n->swsi, WSI_TOKEN_HTTP_COLON_PATH) ||
		    !lws_hdr_total_length(h2n->swsi, WSI_TOKEN_HTTP_COLON_METHOD) ||
		    !lws_hdr_total_length(h2n->swsi, WSI_TOKEN_HTTP_COLON_SCHEME) ||
		     lws_hdr_total_length(h2n->swsi, WSI_TOKEN_HTTP_COLON_STATUS) ||
		     lws_hdr_extant(h2n->swsi, WSI_TOKEN_CONNECTION)) {
			lws_h2_goaway(wsi, H2_ERR_PROTOCOL_ERROR,
				      "Pseudoheader checks");
			break;
		}

So there is no issue.  But show Coverity what it wants so we don't keep getting this
false positive reported by different coverity users.
2020-08-14 06:35:31 +01:00
Andy Green
1a93e73402 fakewsi: replace with smaller substructure
Currently we always reserve a fakewsi per pt so events that don't have a related actual
wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks
that look reasonable to user protocol handler code expecting a valid wsi every time.

This patch splits out stuff that user callbacks often unconditionally expect to be in
a wsi, like context pointer, vhost pointer etc into a substructure, which is composed
into struct lws at the top of it.  Internal references (struct lws is opaque, so there
are only internal references) are all updated to go via the substructre, the compiler
should make that a NOP.

Helpers are added when fakewsi is used and referenced.

If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before,
although the helpers improve consistency by zeroing down the substructure.  There is
a huge amount of user code out there over the last 10 years that did not always have
the minimal examples to follow, some of it does some unexpected things.

If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of
being able to follow the minimal examples' approach.  For PLAT_FREERTOS we don't
reserve the fakewsi in the pt any more, saving around 800 bytes.  The helpers then
create a struct lws_a (the substructure) on the stack, zero it down (but it is only
like 4 pointers) and prepare it with whatever we know like the context.

Then we cast it to a struct lws * and use it in the user protocol handler call.
In this case, the remainder of the struct lws is undefined.  However the amount of
old protocol handlers that might touch things outside of the substructure in
PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is
significant on constrained devices.

User handlers should not be touching everything in a wsi every time anyway, there
are several cases where there is no valid wsi to do the call with.  Dereference of
things outside the substructure should only happen when the callback reason shows
there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-20 06:28:52 +01:00
Andy Green
d339505f51 h2: cmake: build with NO_SERVER 2020-04-16 06:59:45 +01:00
Andy Green
5e99b9f3ee h2: take care to retain initial sid 2020-04-06 13:22:01 +01:00
Andy Green
1c6735e1a0 mux: disable already optimization
This leads to problems at the moment with sticky mux.requested_POLLOUT
causing writeable to not be sent.

Remove it and always set writeable on parents for now.
2020-03-06 07:55:05 +00:00
Andy Green
ac1229f2f7 minimal-http-client-multi: add POST
This adds support for POST in both h1 and h2 queues / stream binding.

The previous queueing tried to keep the "leader" wsi who made the
actual connection around and have it act on the transaction queue
tail if it had done its own thing.

This refactors it so instead, who is the "leader" moves down the
queue and the queued guys inherit the fd, SSL * and queue from the
old leader as they take over.

This lets them operate in their own wsi identity directly and gets
rid of all the "effective wsi" checks, which was applied incompletely
and getting out of hand considering the separate lws_mux checks for
h2 and other muxed protocols alongside it.

This change also allows one wsi at a time to own the transaction for
POST.  --post is added as an option to lws-minimal-http-client-multi
and 6 extra selftests with POST on h1/h2, pipelined or not and
staggered or not are added to the CI.
2020-02-21 17:32:41 +00:00
Andy Green
271ca836c8 event lib: update http client multi to work with it and clean destroy flow
Add selectable event lib support to minimal-http-client-multi and
clean up context destroy flow so we can use lws_destroy_context() from
inside the callback to indicate we want to end the event loop, without
using the traditional "interrupted" flag and in a way that works no
matter which event loop backend is being used.
2020-02-04 14:16:18 +00:00
Andy Green
774240f73b linkit: support build using public sdk
This provides support to build lws using the linkit 7697 public SDK
from here https://docs.labs.mediatek.com/resource/mt7687-mt7697/en/downloads

This toolchain has some challenges, its int32_t / uint32_t are long,
so assumptions about format strings for those being %u / %d / %x all
break.  This fixes all the cases for the features enabled by the
default cmake settings.
2020-01-17 07:45:34 +00:00
Andy Green
6f2230a993 role structs to const
Indicate these are immutable (they're already treated as
such) and can go in .rodata
2020-01-15 06:31:19 +00:00
Andy Green
157acfc906 windows: clean type warnings
There are some minor public api type improvements rather than cast everywhere
inside lws and user code to work around them... these changed from int to
size_t

 - lws_buflist_use_segment() return
 - lws_tokenize_t .len and .token_len
 - lws_tokenize_cstr() length
 - lws_get_peer_simple() namelen
 - lws_get_peer_simple_fd() namelen, int fd -> lws_sockfd_type fd
 - lws_write_numeric_address() len
 - lws_sa46_write_numeric_address() len

These changes are typically a NOP for user code
2020-01-11 14:04:50 +00:00
Andy Green
0bfd39135e cleaning 2020-01-05 22:17:58 +00:00
Andy Green
9cb4f25476 h2: LCCSCF_H2_MANUAL_RXFLOW and refactor txcr
This changes the approach of tx credit management to set the
initial stream tx credit window to zero.  This is the only way
with RFC7540 to gain the ability to selectively precisely rx
flow control incoming streams.

At the time the headers are sent, a WINDOW_UPDATE is sent with
the initial tx credit towards us for that specific stream.  By
default, this acts as before with a 256KB window added for both
the stream and the nwsi, and additional window management sent
as stuff is received.

It's now also possible to set a member in the client info
struct and a new option LCCSCF_H2_MANUAL_RXFLOW to precisely
manage both the initial tx credit for a specific stream and
the ongoing rate limit by meting out further tx credit
manually.

Add another minimal example http-client-h2-rxflow demonstrating how
to force a connection's peer's initial budget to transmit to us
and control it during the connection lifetime to restrict the amount
of incoming data we have to buffer.
2020-01-02 08:31:02 +00:00
Andy Green
7221bc57b5 mux children: generalize helpers out of h2 implementation
This should be a NOP for h2 support and only affects internal
apis.  But it lets us reuse the working and reliable h2 mux
arrangements directly in other protocols later, and share code
so building for h2 + new protocols can take advantage of common
mux child handling struct and code.

Break out common mux handling struct into its own type.

Convert all uses of members that used to be in wsi->h2 to wsi->mux

Audit all references to the members and break out generic helpers
for anything that is useful for other mux-capable protocols to
reuse wsi->mux related features.
2019-12-29 19:59:16 +00:00
Andy Green
37c198d40b format strings: fixes for when toolchain has uint32_t as unsigned long 2019-11-05 21:00:45 +00:00
Andy Green
da2d585583 rtos: h2 in toolchain with long uint32_t 2019-10-17 11:08:47 +01:00
Andy Green
a83c6439e6 client h2: auth bearer option flag
This affects max header size since we use the latter half
of the pt_serv_buf to prepare the (possibly huge) auth token.

Adapt the pt_serv_buf_size in the hugeurl example.
2019-10-12 12:41:14 +01:00
Andy Green
abea7da587 h2: we dont support PUSH 2019-10-12 12:41:14 +01:00
Andy Green
f9f6bb66fe lws_validity: unified connection validity tracking
Refactor everything around ping / pong handling in ws and h2, so there
is instead a protocol-independent validity lws_sul tracking how long it
has been since the last exchange that confirms the operation of the
network connection in both directions.

Clean out periodic role callback and replace the last two role users
with discrete lws_sul for each pt.
2019-09-22 09:35:07 -07:00
Andy Green
ba754c4cb2 h2: unify immortal stream tracking across SSE and ws substreams
It was already correct but add helpers to isolate and deduplicate
processing adding and closing a generically immortal stream.

Change the default 31s h2 network connection timeout to be settable
by .keepalive_timeout if nonzero.

Add a public api allowing a client h2 stream to transition to
half-closed LOCAL (by sending a 0-byte DATA with END_STREAM) and
mark itself as immortal to create a read-only long-poll stream
if the server allows it.

Add a vhost server option flag LWS_SERVER_OPTION_VH_H2_HALF_CLOSED_LONG_POLL
which allows the vhost to treat half-closed remotes as immortal long
poll streams.
2019-09-22 03:08:36 -07:00
Andy Green
6710279e21 client: use block parse and buflist
With http, the protocol doesn't indicate where the headers end and the
next transaction or body begin.  Until now, we handled that for client
header response parsing by reading from the tls buffer bytewise.

This modernizes the code to read in up to 256-byte chunks and parse
the chunks in one hit (the parse API is already set up for doing this
elsewhere).

Now we have a generic input buflist, adapt the parser loop to go through
that and arrange that any leftovers are placed on there.
2019-09-22 03:08:36 -07:00
Andy Green
78c7b0651e buflist: add static reason logging to internal aware apis 2019-09-22 03:08:36 -07:00
Andy Green
32a35d0c4b fixes: various small fixes 2019-09-22 03:08:36 -07:00
Andy Green
49f78ed0d7 client: improve redirect 2019-09-22 03:07:57 -07:00
Andy Green
0fa5563d18 freertos: rename esp32 plat to freertos 2019-08-26 09:58:57 +01:00
Andy Green
ae69bfbd10 debloat: remove things from being built by default that should be conditional 2019-08-26 09:58:57 +01:00
Andy Green
c36a1e8ed0 clean: internally use LWS_WITH_CLIENT and _SERVER
Remove some more things in LWS_WITH_SERVER=0 case
2019-08-26 09:58:57 +01:00
Andy Green
c099e7be92 client: do client stash in a single alloc
Improve the code around stash, getting rid of the strdups for a net
code reduction.  Remove the special destroy helper for stash since
it becomes a one-liner.

Trade several stack allocs in the client reset function for a single
sized brief heap alloc to reduce peak stack alloc by around 700 bytes.
2019-08-19 10:12:20 +01:00
Andy Green
d7f0521aeb private.h: rename to contain dir
Having unique private header names is a requirement of a particular
platform build system it's desirable to work with
2019-08-15 10:49:52 +01:00
Andy Green
26319663f7 license: switch LGPLv2.1+SLE parts to MIT 2019-08-14 10:44:38 +01:00
Andy Green
498a4e2bd7 sul: all timed objects use a single pt sul list
wsi timeout, wsi hrtimer, sequencer timeout and vh-protocol timer
all now participate on a single sorted us list.

The whole idea of polling wakes is thrown out, poll waits ignore the
timeout field and always use infinite timeouts.

Introduce a public api that can schedule its own callback from the event
loop with us resolution (usually ms is all the platform can do).

Upgrade timeouts and sequencer timeouts to also be able to use us resolution.

Introduce a prepared fakewsi in the pt, so we don't have to allocate
one on the heap when we need it.

Directly handle vh-protocol timer if LWS_MAX_SMP == 1
2019-08-09 10:12:09 +01:00
Andy Green
fc5defdd2a COVA10299: check lws_change_pollfd 2019-07-13 13:39:50 -07:00
Andy Green
cbd58fa458 http: body: make sure to consume body before transaction complete
https://github.com/warmcat/libwebsockets/issues/1625

"dead bodies" that were sent but not processed by lws as server
will clog up and destroy transaction tracking if repeated POSTs
with keepalive are sent to nonexistant paths.

This patch introduces a DISCARD_BODY state that follows BODY
except the payload is not signalled to the protocol callback.
Calling transaction_completed() with pending body makes lws
enter DISCARD_BODY and retry transaction completed only after
the pending body is exhausted.
2019-07-05 07:31:38 +01:00
Andy Green
75ef709ff7 buflist: ensure all use callers have nonzero len 2019-07-01 05:53:08 +01:00
Orgad Shaneh
19049d2f28 Change some struct members to unsigned char
Enables removal of superfluous casts, and fixes strict-aliasing warnings with
GCC 4.1.
2019-05-30 08:21:33 +08:00
Andy Green
173943a405 h2: align h1 upgrade to work same as alpn upgrade
https://github.com/warmcat/libwebsockets/issues/1571

Although the code exists for non-tls h1 upgrade to h2, it hasn't been looked
after since all expected uses for h2 are going to be via h2 / alpn.

This patch aligns its upgrade actions with alpn upgrade path so it works OK
via

$ curl --http2 http://localhost:7681/ -v -w "\n"

ie, without tls.  Operation via tls is unaffected.

To use the non-tls upgrade path, you have to be listening without tls, ie with the
test server without -s.  If you're listening in a way that requires tls, this
can't be used to bypass that (or, eg, client certs) in itself, since you have to be
able to talk to it in h1 in the first place to attempt the upgrade to h2.

The common h2 path has some code to dropping the ah unconditionally it looks
like after the first service... this is too aggressive since the first thing
coming on the upgrade path is WINDOW_UPDATE.  It looks wrong anyway, transaction /
stream completion will drop the ah and should be enough.
2019-05-12 08:01:50 +01:00
Andy Green
c13ad5b648 rx flow: use dll2 2019-04-21 19:35:18 +01:00
Andy Green
da3d8cb593 http proxy: support POST 2019-03-22 11:25:22 +08:00
Andy Green
462847bb6f lws_dll: remove lws_dll_lws and deprecate lws_dll_remove 2019-03-21 06:19:31 +08:00
Andy Green
1d466f332e lws_http_mark_sse
https://github.com/warmcat/libwebsockets/issues/1486
2019-01-30 20:59:56 +08:00
Andy Green
08b5ad9299 role: raw-proxy 2018-12-01 11:05:59 +08:00
Andy Green
9bed6d6fc6 clean: general whitespace cleanup 2018-11-23 08:47:56 +08:00
Andy Green
d9f982a055 minimal examples: update for CSP best practices
1) update the logos to svg

2) add svg icon for strict security policy where used

3) define new vhost option flag to enforce sending CSP headers
   with the result code

4) add vhost option flag to minimal examples to
   enforce sending CSP where applicable

5) Go through all the affecting examples confirming they
   still work

6) add LWS_RECOMMENDED_MIN_HEADER_SPACE constant (currently
   2048) to clarify when we need a buffer to hold headers...
   with CSP the headers have become potentially a lot
   larger.
2018-11-21 17:03:29 +08:00
Andy Green
aa4143aebd lws_diskcache: split generic parts from gitohashi into lws 2018-11-12 15:24:42 +08:00
Andy Green
33a6034875 codacy: minor fixes 2018-10-13 08:16:27 +08:00
Andy Green
fd810f198a http proxy: client: unix socket support
This allows the client stuff to understand that addresses beginning with '+'
represent unix sockets.

If the first character after the '+' is '@', it understands that the '@'
should be read as '\0', in order to use Linux "abstract namespace"
sockets.

Further the lws_parse_uri() helper is extended to understand the convention
that an address starting with + is a unix socket, and treats the socket
path as delimited by ':', eg

http://+/var/run/mysocket:/my/path

HTTP Proxy is updated to allow mounts to these unix socket paths.

Proxy connections go out on h1, but are dynamically translated to h1 or h2
on the incoming side.

Proxy usage of libhubbub is separated out... LWS_WITH_HTTP_PROXY is on by
default, and LWS_WITH_HUBBUB is off by default.
2018-09-12 13:58:13 +08:00
Andy Green
ebed5e74cb threadpool 2018-09-11 18:27:59 +08:00
Andy Green
a27dfb2a3e clean: coverity 2018-09-11 18:27:59 +08:00
Andy Green
253942ca80 clean: solve type conversion warnings for appveyor 2018-09-11 18:27:59 +08:00
Andy Green
d58828692e http: compression methods
Add generic http compression layer eanbled at cmake with LWS_WITH_HTTP_STREAM_COMPRESSION.

This is wholly a feature of the HTTP role (used by h1 and h2 roles) and doesn't exist
outside that context.

Currently provides 'deflate' and 'br' compression methods for server side only.

'br' requires also -DLWS_WITH_HTTP_BROTLI=1 at cmake and the brotli libraries (available in
your distro already) and dev package.

Other compression methods can be added nicely using an ops struct.

The built-in file serving stuff will use this is the client says he can handle it, and the
mimetype of the file either starts with "text/" (html and css etc) or is the mimetype of
Javascript.

zlib allocates quite a bit while in use, it seems to be around 256KiB per stream.  So this
is only useful on relatively strong servers with lots of memory.  However for some usecases
where you are serving a lot of css and js assets, it's a nice help.

The patch performs special treatment for http/1.1 pipelining, since the compression is
performed on the fly the compressed content-length is not known until the end.  So for h1
only, chunked transfer-encoding is automatically added so pipelining can continue of the
connection.

For h2 the chunking is neither supported nor required, so it "just works".

User code can also request to add a compression transform before the reply headers were
sent using the new api

LWS_VISIBLE int
lws_http_compression_apply(struct lws *wsi, const char *name,
			   unsigned char **p, unsigned char *end, char decomp);

... this allows transparent compression of dynamically generated HTTP.  The requested
compression (eg, "deflate") is only applied if the client headers indicated it was
supported, otherwise it's a NOP.

Name may be NULL in which case the first compression method in the internal table at
stream.c that is mentioned as acceptable by the client will be used.

NOTE: the compression translation, same as h2 support, relies on the user code using
LWS_WRITE_HTTP and then LWS_WRITE_HTTP_FINAL on the last part written.  The internal
lws fileserving code already does this.
2018-09-02 14:43:05 +08:00