1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-16 00:00:07 +01:00
Commit graph

154 commits

Author SHA1 Message Date
Andy Green
3b44a745c9 h2-tx-credit-disallow-overflow 2019-07-08 08:48:58 +01:00
Andy Green
55599fef71 h2: SETTINGS: allow 16M-1 peer max frame size
We only allow 8M-1 when we should allow 16M-1 without blowing a
PROTOCOL ERROR.
2019-07-07 12:19:54 +01:00
Andy Green
cbd58fa458 http: body: make sure to consume body before transaction complete
https://github.com/warmcat/libwebsockets/issues/1625

"dead bodies" that were sent but not processed by lws as server
will clog up and destroy transaction tracking if repeated POSTs
with keepalive are sent to nonexistant paths.

This patch introduces a DISCARD_BODY state that follows BODY
except the payload is not signalled to the protocol callback.
Calling transaction_completed() with pending body makes lws
enter DISCARD_BODY and retry transaction completed only after
the pending body is exhausted.
2019-07-05 07:31:38 +01:00
Andy Green
75ef709ff7 buflist: ensure all use callers have nonzero len 2019-07-01 05:53:08 +01:00
Chen Xi
892cde2c71 rtos: add support for AMAZON_RTOS 2019-06-05 05:04:17 +01:00
Orgad Shaneh
19049d2f28 Change some struct members to unsigned char
Enables removal of superfluous casts, and fixes strict-aliasing warnings with
GCC 4.1.
2019-05-30 08:21:33 +08:00
Andy Green
173943a405 h2: align h1 upgrade to work same as alpn upgrade
https://github.com/warmcat/libwebsockets/issues/1571

Although the code exists for non-tls h1 upgrade to h2, it hasn't been looked
after since all expected uses for h2 are going to be via h2 / alpn.

This patch aligns its upgrade actions with alpn upgrade path so it works OK
via

$ curl --http2 http://localhost:7681/ -v -w "\n"

ie, without tls.  Operation via tls is unaffected.

To use the non-tls upgrade path, you have to be listening without tls, ie with the
test server without -s.  If you're listening in a way that requires tls, this
can't be used to bypass that (or, eg, client certs) in itself, since you have to be
able to talk to it in h1 in the first place to attempt the upgrade to h2.

The common h2 path has some code to dropping the ah unconditionally it looks
like after the first service... this is too aggressive since the first thing
coming on the upgrade path is WINDOW_UPDATE.  It looks wrong anyway, transaction /
stream completion will drop the ah and should be enough.
2019-05-12 08:01:50 +01:00
Andy Green
c13ad5b648 rx flow: use dll2 2019-04-21 19:35:18 +01:00
Andy Green
da3d8cb593 http proxy: support POST 2019-03-22 11:25:22 +08:00
Andy Green
d1d313b4bf ws proxy: also proxy h1 ws to h1 and h2
lws has been able to proxy h2 or h1 inbound connections to an
h1 onward connection for a while now.  It's simple to use just
build with LWS_WITH_HTTP_PROXY and make a mount where the origin
is the onward connection details.  Unix sockets can also be
used as the onward connection.

This patch extends the support to be able to also do the same for
inbound h2 or h1 ws upgrades to an h1 ws onward connection as well.

This allows you to offer completely different services in a
common URL space, including ones that connect back by ws / wss.
2019-03-21 10:26:47 +08:00
Andy Green
462847bb6f lws_dll: remove lws_dll_lws and deprecate lws_dll_remove 2019-03-21 06:19:31 +08:00
Andy Green
986bb37c88 ah: custom headers for h1
Until now lws only parses headers it knows at build-time from its
prebuilt lexical analyzer.

This adds an on-by-default cmake option and a couple of apis
to also store and query "custom", ie, unknown-to-lws headers.

A minimal example is also provided.

At the moment it only works on h1, h2 support needs improvements
to the hpack implementation.

Since it bloats ah memory usage compared to without it if custom
headers are present, the related code and ah footprint can be
disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's
on by default normally.  ESP32 platform disables it.

https://github.com/warmcat/libwebsockets/pull/1499
2019-03-10 08:02:02 +08:00
Andy Green
1d466f332e lws_http_mark_sse
https://github.com/warmcat/libwebsockets/issues/1486
2019-01-30 20:59:56 +08:00
Andy Green
08b5ad9299 role: raw-proxy 2018-12-01 11:05:59 +08:00
Andy Green
43cf8bb391 wsi: opaque_user_data and accessors
Under some circumstances it's useful to tag a wsi with user
data, while still having an lws-allocated and destroyed pss.
2018-12-01 11:05:59 +08:00
Andy Green
9bed6d6fc6 clean: general whitespace cleanup 2018-11-23 08:47:56 +08:00
Andy Green
d9f982a055 minimal examples: update for CSP best practices
1) update the logos to svg

2) add svg icon for strict security policy where used

3) define new vhost option flag to enforce sending CSP headers
   with the result code

4) add vhost option flag to minimal examples to
   enforce sending CSP where applicable

5) Go through all the affecting examples confirming they
   still work

6) add LWS_RECOMMENDED_MIN_HEADER_SPACE constant (currently
   2048) to clarify when we need a buffer to hold headers...
   with CSP the headers have become potentially a lot
   larger.
2018-11-21 17:03:29 +08:00
Andy Green
66b73c680c coverity 192910: check length in h2 header dump
Since the NUL was always taken care of at the end of buf, the worst it could
do was log some junk.  This makes it explicitly log it as oversize separately.
2018-11-13 16:53:41 +08:00
Andy Green
aa4143aebd lws_diskcache: split generic parts from gitohashi into lws 2018-11-12 15:24:42 +08:00
Andy Green
cbb8b1d3e9 LWS_WITH_ASAN
This lets you build using the runtime Address Sanitizer in gcc.

LWS is heavily tested with valgrind routinely during development.  But ASAN
did find some theoretical-only issues with shifting, strictly ~(1 << 31) is
a signed int, it should be ~(1u << 31).  Gcc does the same for both, but it's
good to have the ability to find these.
2018-11-03 14:47:48 +08:00
Andy Green
33a6034875 codacy: minor fixes 2018-10-13 08:16:27 +08:00
Andy Green
5760a562d8 ws-over-h2: unbreak 2018-10-02 05:42:56 +08:00
Andy Green
fd810f198a http proxy: client: unix socket support
This allows the client stuff to understand that addresses beginning with '+'
represent unix sockets.

If the first character after the '+' is '@', it understands that the '@'
should be read as '\0', in order to use Linux "abstract namespace"
sockets.

Further the lws_parse_uri() helper is extended to understand the convention
that an address starting with + is a unix socket, and treats the socket
path as delimited by ':', eg

http://+/var/run/mysocket:/my/path

HTTP Proxy is updated to allow mounts to these unix socket paths.

Proxy connections go out on h1, but are dynamically translated to h1 or h2
on the incoming side.

Proxy usage of libhubbub is separated out... LWS_WITH_HTTP_PROXY is on by
default, and LWS_WITH_HUBBUB is off by default.
2018-09-12 13:58:13 +08:00
Andy Green
ebed5e74cb threadpool 2018-09-11 18:27:59 +08:00
Andy Green
e618ce1d4c h2: use vhost headers with tolower adaptation
h2 was not applying vhost headers, make it do so.

Also adapt any header names to lower-case as required by h2.
2018-09-11 18:27:59 +08:00
Andy Green
a27dfb2a3e clean: coverity 2018-09-11 18:27:59 +08:00
Andy Green
253942ca80 clean: solve type conversion warnings for appveyor 2018-09-11 18:27:59 +08:00
Andy Green
d58828692e http: compression methods
Add generic http compression layer eanbled at cmake with LWS_WITH_HTTP_STREAM_COMPRESSION.

This is wholly a feature of the HTTP role (used by h1 and h2 roles) and doesn't exist
outside that context.

Currently provides 'deflate' and 'br' compression methods for server side only.

'br' requires also -DLWS_WITH_HTTP_BROTLI=1 at cmake and the brotli libraries (available in
your distro already) and dev package.

Other compression methods can be added nicely using an ops struct.

The built-in file serving stuff will use this is the client says he can handle it, and the
mimetype of the file either starts with "text/" (html and css etc) or is the mimetype of
Javascript.

zlib allocates quite a bit while in use, it seems to be around 256KiB per stream.  So this
is only useful on relatively strong servers with lots of memory.  However for some usecases
where you are serving a lot of css and js assets, it's a nice help.

The patch performs special treatment for http/1.1 pipelining, since the compression is
performed on the fly the compressed content-length is not known until the end.  So for h1
only, chunked transfer-encoding is automatically added so pipelining can continue of the
connection.

For h2 the chunking is neither supported nor required, so it "just works".

User code can also request to add a compression transform before the reply headers were
sent using the new api

LWS_VISIBLE int
lws_http_compression_apply(struct lws *wsi, const char *name,
			   unsigned char **p, unsigned char *end, char decomp);

... this allows transparent compression of dynamically generated HTTP.  The requested
compression (eg, "deflate") is only applied if the client headers indicated it was
supported, otherwise it's a NOP.

Name may be NULL in which case the first compression method in the internal table at
stream.c that is mentioned as acceptable by the client will be used.

NOTE: the compression translation, same as h2 support, relies on the user code using
LWS_WRITE_HTTP and then LWS_WRITE_HTTP_FINAL on the last part written.  The internal
lws fileserving code already does this.
2018-09-02 14:43:05 +08:00
Andy Green
90e6e65bff partial: replace ad-hoc code with a wsi buflist_out
Various kinds of input stashing were replaced with a single buflist before
v3.0... this patch replaces the partial send arrangements with its own buflist
in the same way.

Buflists as the name says are growable lists of allocations in a linked-list
that take care of book-keeping what's added and removed (even if what is
removed is less than the current buffer on the list).

The immediate result is that we no longer have to freak out if we had a partial
buffered and new output is coming... we can just pile it on the end of the
buflist and keep draining the front of it.

Likewise we no longer need to be rabid about reporting multiple attempts to
send stuff without going back to the event loop, although not doing that
will introduce inefficiencies we don't have to term it "illegal" any more.

Since buflists have proven reliable on the input side and the logic for dealing
with truncated "non-network events" was already there this internal-only change
should be relatively self-contained.
2018-08-20 12:02:26 +08:00
Andy Green
5c0b0450f2 client: bind and drop protocol like server
HTTP server protocols have had for a while LWS_CALLBACK_HTTP_DROP/BIND_PROTOCOL
callbacks that mark when a wsi is attched to a protocol and detached.

It turns out this is generally useful for everything to know when a wsi is
joining a protocol and definitively completely finished with a protocol.

Particularly with client wsi where you provided the userdata externally, this
makes a clear point to free() it on the protocol binding being dropped.

This patch adds protocol bind / unbind callbacks to the role definition and
lets them operate on all roles.  For the various roles

HTTP server: LWS_CALLBACK_HTTP_BIND/DROP_PROTOCOL as before
HTTP client: LWS_CALLBACK_CLIENT_HTTP_BIND/DROP_PROTOCOL
ws server:   LWS_CALLBACK_WS_SERVER_BIND/DROP_PROTOCOL
ws client:   LWS_CALLBACK_WS_CLIENT_BIND/DROP_PROTOCOL
raw file:    LWS_CALLBACK_RAW_FILE_BIND/DROP_PROTOCOL
raw skt:     LWS_CALLBACK_RAW_SKT_BIND/DROP_PROTOCOL
2018-08-18 14:11:29 +08:00
Andy Green
d461f46a97 libwebsockets.h: clean out some boilerplate better put in core/private.h
https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
Andy Green
a03dd40e62 plugins: gitws libjsongit2 support
This adds a plugin that interfaces to libjsongit2

https://warmcat.com/git/libjsongit2

to provide a per-vhost service for presenting bare git repos in a
web interface.
2018-08-14 08:00:25 +08:00
Andy Green
ac3bd36c60 vhost_destroy: use vhost wsi reference counting to trigger destroy
This changes the vhost destroy flow to only hand off the listen
socket if another vhost sharing it, and mark the vhost as
being_destroyed.

Each tsi calls lws_check_deferred_free() once a second, if it sees
any vhost being_destroyed there, it closes all wsi on its tsi on
the same vhost, one time.

As the wsi on the vhost complete close (ie, after libuv async close
if on libuv event loop), they decrement a reference count for all
wsi open on the vhost.  The tsi who closes the last one then
completes the destroy flow for the vhost itself... it's random
which tsi completes the vhost destroy but since there are no
wsi left on the vhost, and it holds the context lock, nothing
can conflict.

The advantage of this is that owning tsi do the close for wsi
that are bound to the vhost under destruction, at a time when
they are guaranteed to be idle for service, and they do it with
both vhost and context locks owned, so no other service thread
can conflict for stuff protected by those either.

For the situation the user code may have allocations attached to
the vhost, this adds args to lws_vhost_destroy() to allow destroying
the user allocations just before the vhost is freed.
2018-06-18 09:11:46 +08:00
Andy Green
502130d999 refactor: split out adoption and client apis to core
- split raw role into separate skt and file

 - remove all special knowledge from the adoption
   apis and migrate to core

 - remove all special knowledge from client_connect
   stuff, and have it discovered by iterating the
   role callbacks to let those choose how to bind;
   migrate to core

 - retire the old deprecated client apis pre-
   client_connect_info
2018-05-11 10:29:08 +08:00
Andy Green
de064fd65a refactor: core code in lib/core and private-libwebsockets.h to core/private.h
This commit is coverity-clean as tested

cmake .. -DLWS_WITH_MINIMAL_EXAMPLES=1 -DLWS_WITHOUT_EXTENSIONS=1 -DLWS_WITH_ACME=1 -DLWS_WITH_LWSWS=1 -DLWS_WITH_LIBUV=1 -DLWS_WITH_HTTP2=1 -DLWS_WITHOUT_CLIENT=0 -DLWS_WITHOUT_SERVER=0 -DLWS_UNIX_SOCK=1 -DLWS_WITH_TLS=0 -DLWS_WITH_MBEDTLS=0 -DLWS_WITH_CGI=1 -DCMAKE_BUILD_TYPE=DEBUG -DLWS_WITH_GENERIC_SESSIONS=1 -DLWS_WITH_RANGES=1 -DLWS_ROLE_WS=1 -DLWS_MAX_SMP=16 -DLWS_ROLE_H1=1 -DLWS_WITH_WOLFSSL=0 -DLWS_WITH_LIBEV=0 -DLWS_WITH_LIBEVENT=1
2018-05-03 10:49:36 +08:00
Andy Green
da0be64f68 minimal-raw-netcat
Adapt attack.sh to use it instead of OS netcat and fox various bugs that
exposed.
2018-05-02 08:46:16 +08:00
Andy Green
f497562a62 refactor: change event lib minimal examples to serve as the test apps 2018-04-30 19:17:32 +08:00
Andy Green
d37b383edc refactor: apply ops structs to event loop handlers 2018-04-29 10:44:36 +08:00
Andy Green
ac6c48d98f refactor: most preparation for -DLWS_ROLE_H1=0 2018-04-27 19:16:50 +08:00
Andy Green
16a907180c refactor: move all ah and http specific vars to http private and conditional inclusion from there 2018-04-27 15:20:56 +08:00
Andy Green
7c0a2ae633 smp: h2 children must go on same tsi as parent
We can't have a situation where stream wsis inside an h2 connection
are serviced by different threads than the actual parent connection.
2018-04-27 12:49:42 +08:00
Andy Green
7ff8f023d1 context info struct: make lws usage all const 2018-04-27 08:37:20 +08:00
Andy Green
5d06f610a9 travis: h2spec integration 2018-04-26 15:27:02 +08:00
Andy Green
2e3ddeedb6 lws_get_peer_write_allowance: return lws_fileofs_t 2018-04-26 08:30:12 +08:00
Andy Green
27e86e2641 cmake: allow setting LWS_ROLE_WS
You can build lws without support for ws, with -DLWS_ROLE_WS=0.

This is thanks to the role ops isolating all the ws-specific business
in the ws role.

Also retire more test apps replaced by minmal-examples.
2018-04-25 08:42:18 +08:00
Andy Green
8e9751e26e refactor: move out extension lws members into ws and establish role structs for vhost content 2018-04-25 06:53:30 +08:00
Andy Green
7b227eb333 autobahn fixes
This replaces the old test-app for echo with separate client and server
minimal versions.

The autobahn test script is made more autonomous and tests both
client and server.
2018-04-22 06:45:46 +08:00
Andy Green
3f683351b3 refactor: split out private role header content
Private header stuff specific to roles should go in the
role dir and only be included if the role is enabled for
build.

Only definitions related to lws core should go in the actual
private-libwebsockets.h
2018-04-20 07:13:05 +08:00
Andy Green
1d2094996e refactor: absorb other caches into buflist
1) Remove the whole ah rxbuf and put things on to the wsi buflist

This eliminates the whole detachability thing based on ah rxbuf
state... ah can always be detached.

2) Remove h2 scratch and put it on the wsi buflist

3) Remove preamble_rx and use the wsi buflist

This was used in the case adopted sockets had already been read.


Basically there are now only three forced service scenarios

 - something in buflist (and not in state LRS_DEFERRING_ACTION)

 - tls layer has buffered rx

 - extension has buffered rx

This is a net removal of around 400 lines of special-casing.
2018-04-20 07:13:05 +08:00
Andy Green
8829c2f365 selftests
This converts several of the selftests to return a status in their exit code
about whether they 'worked'.

A small bash script framework is added, with a selftest.sh in the mininmal
example dirs that support it, and a ./minimal-examples/selftests.sh script
that can be run from the build dir with no args that discovers and runs all
the selftest.sh scripts underneath.

That is also integrated into travis and the enabled tests must pass now for
travis to pass.  Travis does not have a modern libuv so it can't run a
couple of tests which are nulled out if it sees it's running in travis env.
2018-04-19 16:16:48 +08:00