1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-16 00:00:07 +01:00
Commit graph

59 commits

Author SHA1 Message Date
Andy Green
28c0e51cd6 client: unify post tls accept handling 2020-03-07 20:20:07 +00:00
Andy Green
28ce32af64 client: secure streams
Secure Streams is an optional layer on top of lws that separates policy
like endpoint selection and tls cert validation into a device JSON
policy document.

Code that wants to open a client connection just specifies a streamtype name,
and no longer deals with details like the endpoint, the protocol (!) or anything
else other than payloads and optionally generic metadata; the JSON policy
contains all the details for each streamtype.  h1, h2, ws and mqtt client
connections are supported.

Logical secure streams outlive any particular connection and supports "nailed-up"
connectivity regardless of underlying connection stability.
2020-03-04 12:17:49 +00:00
Andy Green
d88d41090e cleaning: windows 2020-03-04 12:17:49 +00:00
Andy Green
bb1a8d37fb rtos diet: make basic auth support a config option on by default
Remove wrapper content for < tls v1.2
2020-03-04 11:00:04 +00:00
Andy Green
0dd23d7073 tls restriction: account for pipelining and add test 2020-03-01 08:14:43 +00:00
Andy Green
af2f1f9a87 event-lib-glib
Adds support for glib event library.
2020-02-21 17:32:41 +00:00
Andy Green
ac1229f2f7 minimal-http-client-multi: add POST
This adds support for POST in both h1 and h2 queues / stream binding.

The previous queueing tried to keep the "leader" wsi who made the
actual connection around and have it act on the transaction queue
tail if it had done its own thing.

This refactors it so instead, who is the "leader" moves down the
queue and the queued guys inherit the fd, SSL * and queue from the
old leader as they take over.

This lets them operate in their own wsi identity directly and gets
rid of all the "effective wsi" checks, which was applied incompletely
and getting out of hand considering the separate lws_mux checks for
h2 and other muxed protocols alongside it.

This change also allows one wsi at a time to own the transaction for
POST.  --post is added as an option to lws-minimal-http-client-multi
and 6 extra selftests with POST on h1/h2, pipelined or not and
staggered or not are added to the CI.
2020-02-21 17:32:41 +00:00
Andy Green
271ca836c8 event lib: update http client multi to work with it and clean destroy flow
Add selectable event lib support to minimal-http-client-multi and
clean up context destroy flow so we can use lws_destroy_context() from
inside the callback to indicate we want to end the event loop, without
using the traditional "interrupted" flag and in a way that works no
matter which event loop backend is being used.
2020-02-04 14:16:18 +00:00
Andy Green
0bfd39135e cleaning 2020-01-05 22:17:58 +00:00
Andy Green
d8ccfc2370 lws_system: helpers for attaching to existing event loop from other threads
In the case code is composed into a single process, but it isn't monolithic in the
sense it's made up of modular "applications" that are written separate projects,
provide a way for the "applications" to request a callback from the lws event loop
thread context safely.

From the callback the applications can set up their operations on the lws event
loop and drop their own thread.

Since it requires system-specific locking to be threadsafe, provide a non-threadsafe
helper and then indirect the actual usage through a user-defined lws_system ops
function pointer that wraps the unsafe api with the system locking to make it safe.
2020-01-05 22:17:58 +00:00
Andy Green
9cb4f25476 h2: LCCSCF_H2_MANUAL_RXFLOW and refactor txcr
This changes the approach of tx credit management to set the
initial stream tx credit window to zero.  This is the only way
with RFC7540 to gain the ability to selectively precisely rx
flow control incoming streams.

At the time the headers are sent, a WINDOW_UPDATE is sent with
the initial tx credit towards us for that specific stream.  By
default, this acts as before with a 256KB window added for both
the stream and the nwsi, and additional window management sent
as stuff is received.

It's now also possible to set a member in the client info
struct and a new option LCCSCF_H2_MANUAL_RXFLOW to precisely
manage both the initial tx credit for a specific stream and
the ongoing rate limit by meting out further tx credit
manually.

Add another minimal example http-client-h2-rxflow demonstrating how
to force a connection's peer's initial budget to transmit to us
and control it during the connection lifetime to restrict the amount
of incoming data we have to buffer.
2020-01-02 08:31:02 +00:00
Andy Green
36de0ada7d http client basic auth add helper and example 2019-12-22 18:17:45 +00:00
Andy Green
fb25b64b83 minimal http client multi: rename struct user so it cant conflict 2019-12-09 14:48:54 +00:00
Andy Green
efbab4c602 minimal client: add --path option 2019-12-03 07:33:19 +00:00
Andy Green
0f7f27801e http redirect: 303: force method to GET
This teaches http client stuff how to handle 303 redirects... these
can happen after POST where the server side wants you to come back with
a GET to the Location: mentioned.

lws client will follow the redirect and force GET, this works for both
h1 and h2.  Client protocol handler has to act differently if it finds
it is connecting for the initial POST or the subsequent GET, it can
find out which by checking a new api lws_http_is_redirected_to_get(wsi)
which returns nonzero if in GET mode.

Minimal example for server form-post has a new --303 switch to enable
this behaviour there and the client post example has additions to
check lws_http_is_redirected_to_get().
2019-11-17 10:47:01 +00:00
Andy Green
0ab4a707fb format strings: ban %.*s as some platforms lack it
The %.*s is very handy to print strings where you have a length, but
there is no NUL termination.  It's quite widely supported but at least
one vendor RTOS toolchain doesn't have it.

Since there aren't that many uses of it yet, audit all uses and
convert to a new helper lws_strnncpy() which uses the smaller of
two lengths.
2019-11-04 13:49:14 +00:00
Andy Green
6a56855284 minimal client http: add lws_get_peer_simple example 2019-11-02 10:11:20 +00:00
Andy Green
127e53cf98 client: multipart mime generation helpers
lws has been able to generate client multipart mime as shown
in minimal-http-client-post, but it requires a lot of user
boilerplate to handle the boundary, related transaction header,
and multipart headers.

This patch adds a client creation flag to indicate it will
carry multipart mime, which autocreates the boundary string
and applies the transaction header with it, and an api to
form the boundary headers between the different mime parts
and the terminating boundary.
2019-10-12 12:41:14 +01:00
Andy Green
a83c6439e6 client h2: auth bearer option flag
This affects max header size since we use the latter half
of the pt_serv_buf to prepare the (possibly huge) auth token.

Adapt the pt_serv_buf_size in the hugeurl example.
2019-10-12 12:41:14 +01:00
Andy Green
eda6707a84 client: tls cert in mem 2019-10-12 12:41:14 +01:00
Andy Green
bb7f96d32b cmdline: introduce builtin switches
Rather than do all switches by hand on the minimal examples,
add a helper that knows some "builtin" ones like -d and
others to set context options you might want to use in
any example.
2019-10-10 16:34:37 +01:00
Andy Green
bce1f01370 lws_state and system state
Introduce a generic lws_state object with notification handlers
that may be registered in a chain.

Implement one of those in the context to manage the "system state".

Allow other pieces of lws and user code to register notification
handlers on a context list.  Handlers can object to or take over
responsibility to move forward and retry system state changes if
they know that some dependent action must succeed first.

For example if the system time is invalid, we cannot move on to
a state where anything can do tls until that has been corrected.
2019-09-22 09:35:07 -07:00
Andy Green
f9f6bb66fe lws_validity: unified connection validity tracking
Refactor everything around ping / pong handling in ws and h2, so there
is instead a protocol-independent validity lws_sul tracking how long it
has been since the last exchange that confirms the operation of the
network connection in both directions.

Clean out periodic role callback and replace the last two role users
with discrete lws_sul for each pt.
2019-09-22 09:35:07 -07:00
Jakob Szumacher
503bb8f8c9 client: LCCSCF_ALLOW_INSECURE
AG add options the minimal http client related to this
2019-09-22 09:35:07 -07:00
Andy Green
ba754c4cb2 h2: unify immortal stream tracking across SSE and ws substreams
It was already correct but add helpers to isolate and deduplicate
processing adding and closing a generically immortal stream.

Change the default 31s h2 network connection timeout to be settable
by .keepalive_timeout if nonzero.

Add a public api allowing a client h2 stream to transition to
half-closed LOCAL (by sending a 0-byte DATA with END_STREAM) and
mark itself as immortal to create a read-only long-poll stream
if the server allows it.

Add a vhost server option flag LWS_SERVER_OPTION_VH_H2_HALF_CLOSED_LONG_POLL
which allows the vhost to treat half-closed remotes as immortal long
poll streams.
2019-09-22 03:08:36 -07:00
Andy Green
0fb976837b tls certs: update warmcat.com and libwebsockets.org public certs
Old certs were getting near the end of their life and we switched the
server to use letsencrypt.  The root and intermediate needed for the
mbedtls case changed accordingly
2019-09-22 03:08:36 -07:00
Andy Green
d808748cd6 detailed latency stats
Remove LWS_LATENCY.

Add the option LWS_WITH_DETAILED_LATENCY, allowing lws to collect very detailed
information on every read and write, and allow the user code to provide
a callback to process events.
2019-09-22 03:06:59 -07:00
Andy Green
c591e1adfc asynchronous dns for ipv4 and ipv6
This adds the option to have lws do its own dns resolution on
the event loop, without blocking.  Existing implementations get
the name resolution done by the libc, which is blocking.  In
the case you are opening client connections but need to carefully
manage latency, another connection opening and doing the name
resolution becomes a big problem.

Currently it supports

 - ipv4 / A records
 - ipv6 / AAAA records
 - ipv4-over-ipv6 ::ffff:1.2.3.4 A record promotion for ipv6
 - only one server supported over UDP :53
 - nameserver discovery on linux, windows, freertos

It also has some nice advantages

 - lws-style paranoid response parsing
 - random unique tid generation to increase difficulty of poisoning
 - it's really integrated with the lws event loop, it does not spawn
   threads or use the libc resolver, and of course no blocking at all
 - platform-specific server address capturing (from /etc/resolv.conf
   on linux, windows apis on windows)
 - it has LRU caching
 - piggybacking (multiple requests before the first completes go on
   a list on the first request, not spawn multiple requests)
 - observes TTL in cache
 - TTL and timeout use lws_sul timers on the event loop
 - ipv6 pieces only built if cmake LWS_IPV6 enabled
2019-09-19 06:54:53 +01:00
Andy Green
c36a1e8ed0 clean: internally use LWS_WITH_CLIENT and _SERVER
Remove some more things in LWS_WITH_SERVER=0 case
2019-08-26 09:58:57 +01:00
Andy Green
498a4e2bd7 sul: all timed objects use a single pt sul list
wsi timeout, wsi hrtimer, sequencer timeout and vh-protocol timer
all now participate on a single sorted us list.

The whole idea of polling wakes is thrown out, poll waits ignore the
timeout field and always use infinite timeouts.

Introduce a public api that can schedule its own callback from the event
loop with us resolution (usually ms is all the platform can do).

Upgrade timeouts and sequencer timeouts to also be able to use us resolution.

Introduce a prepared fakewsi in the pt, so we don't have to allocate
one on the heap when we need it.

Directly handle vh-protocol timer if LWS_MAX_SMP == 1
2019-08-09 10:12:09 +01:00
Andy Green
48366de1d1 unix plat: add minimal wsi fd map option
An lws context usually contains a processwide fd -> wsi lookup table.

This allows any possible fd returned by a *nix type OS to be immediately
converted to a wsi just by indexing an array of struct lws * the size of
the highest possible fd, as found by ulimit -n or similar.

This works modestly for Linux type systems where the default ulimit -n for
a process is 1024, it means a 4KB or 8KB lookup table for 32-bit or
64-bit systems.

However in the case your lws usage is much simpler, like one outgoing
client connection and no serving, this represents increasing waste.  It's
made much worse if the system has a much larger default ulimit -n, eg 1M,
the table is occupying 4MB or 8MB, of which you will only use one.

Even so, because lws can't be sure the OS won't return a socket fd at any
number up to (ulimit -n - 1), it has to allocate the whole lookup table
at the moment.

This patch looks to see if the context creation info is setting
info->fd_limit_per_thread... if it leaves it at the default 0, then
everything is as it was before this patch.  However if finds that
(info->fd_limit_per_thread * actual_number_of_service_threads) where
the default number of service threads is 1, is less than the fd limit
set by ulimit -n, lws switches to a slower lookup table scheme, which
only allocates the requested number of slots.  Lookups happen then by
iterating the table and comparing rather than indexing the array
directly, which is obviously somewhat of a performance hit.

However in the case where you know lws will only have a very few wsi
maximum, this method can very usefully trade off speed to be able to
avoid the allocation sized by ulimit -n.

minimal examples for client that can make use of this are also modified
by this patch to use the smaller context allocations.
2019-05-18 12:10:19 +01:00
Andy Green
f7149e90c2 cc0: align dedication to CC0 FAQ recommended format
https://libwebsockets.org/pipermail/libwebsockets/2019-April/007937.html

thanks to Bruce Perens for noting it.

This doesn't change the intention or status of the CC0 files, they were
pure CC0 before (ie, public domain) and they are pure CC0 now.  It just
gets rid of the (C) part at the top of the dedication which may be read
to be a bit contradictory since the purpose is to make it public domain.
2019-05-02 09:29:01 +01:00
Andy Green
b227d10187 nossl recv: always respond to 0 length read as shutdown 2019-03-21 10:24:58 +08:00
Andy Green
4ed522eb9c minimal-http-client-custom-headers 2019-03-12 11:57:43 +08:00
Andy Green
f8cd973f85 mingw: windows: make minimal examples build 2019-03-10 08:02:02 +08:00
Andy Green
991241905c client: use effective wsi for some callbacks 2018-07-20 08:40:53 +08:00
Andy Green
44e5e1231b minimal client: add different paths to multi 2018-06-04 07:14:49 +08:00
Andy Green
bce8cca042 refactor: also migrate tls to the ops struct and private.h pattern
Several new ops are planned for tls... so better to bite the bullet and
clean it out to the same level as roles + event-libs first.

Also adds a new travis target "mbedtls" and all the tests except
autobahn against mbedtls build.
2018-05-02 12:10:36 +08:00
Andy Green
f497562a62 refactor: change event lib minimal examples to serve as the test apps 2018-04-30 19:17:32 +08:00
Andy Green
ac6c48d98f refactor: most preparation for -DLWS_ROLE_H1=0 2018-04-27 19:16:50 +08:00
Andy Green
800cd40f88 client: support pipelining / h2 multi for POST
https://github.com/warmcat/libwebsockets/issues/1256
2018-04-27 07:25:45 +08:00
Andy Green
1d2094996e refactor: absorb other caches into buflist
1) Remove the whole ah rxbuf and put things on to the wsi buflist

This eliminates the whole detachability thing based on ah rxbuf
state... ah can always be detached.

2) Remove h2 scratch and put it on the wsi buflist

3) Remove preamble_rx and use the wsi buflist

This was used in the case adopted sockets had already been read.


Basically there are now only three forced service scenarios

 - something in buflist (and not in state LRS_DEFERRING_ACTION)

 - tls layer has buffered rx

 - extension has buffered rx

This is a net removal of around 400 lines of special-casing.
2018-04-20 07:13:05 +08:00
Andy Green
8829c2f365 selftests
This converts several of the selftests to return a status in their exit code
about whether they 'worked'.

A small bash script framework is added, with a selftest.sh in the mininmal
example dirs that support it, and a ./minimal-examples/selftests.sh script
that can be run from the build dir with no args that discovers and runs all
the selftest.sh scripts underneath.

That is also integrated into travis and the enabled tests must pass now for
travis to pass.  Travis does not have a modern libuv so it can't run a
couple of tests which are nulled out if it sees it's running in travis env.
2018-04-19 16:16:48 +08:00
Andy Green
f0048acbee minimal: add switches and exit results 2018-04-19 16:16:48 +08:00
Andy Green
aa816e98a9 alpn: assemble defaults from roles and allow override
Since new roles may be incompatible with http, add support for
alpn names at the role struct, automatic generation of the
default list of alpn names that servers advertise, and the
ability to override the used alpn names per-vhost and per-
client connection.

This not only lets you modulate visibility or use of h2,
but also enables vhosts that only offer non-http roles,
as well as restricting http role vhosts to only alpn
identifiers related to http roles.
2018-04-19 16:15:10 +08:00
Andy Green
126be3ccf3 refactor role ops
This only refactors internal architecture and representations, the user
api is unaffected.
2018-04-11 13:39:42 +08:00
Andy Green
16e2f09710 refactor mode and states into roles and states 2018-04-06 12:22:19 +08:00
Andy Green
9a51bd0a63 LWS_TLS_CERT_INFO_OPAQUE_PUBLIC_KEY 2018-04-06 10:38:04 +08:00
Andy Green
719f735309 minimal-http-client-post 2018-04-06 10:38:04 +08:00
Andy Green
2a9b6f54c6 client: h2
This adds h2 http support for the client api.

The public client api requires no changes, it will detect by
ALPN if the server can handle http/2, if so, it will use it.

Multiple client connections using the lws api will be mapped on
to the same single http/2 + tls socket using http/2 streams
that are serviced simultaneously where possible.
2018-04-06 10:38:03 +08:00