wsi timeout, wsi hrtimer, sequencer timeout and vh-protocol timer
all now participate on a single sorted us list.
The whole idea of polling wakes is thrown out, poll waits ignore the
timeout field and always use infinite timeouts.
Introduce a public api that can schedule its own callback from the event
loop with us resolution (usually ms is all the platform can do).
Upgrade timeouts and sequencer timeouts to also be able to use us resolution.
Introduce a prepared fakewsi in the pt, so we don't have to allocate
one on the heap when we need it.
Directly handle vh-protocol timer if LWS_MAX_SMP == 1
There are quite a few linked-lists of things that want events after
some period. This introduces a type binding an lws_dll2 for the
list and a lws_usec_t for the duration.
The wsi timeouts, the hrtimer and the sequencer timeouts are converted
to use these, also in the common event wait calculation.
lws_dll2 removes the downsides of lws_dll and adds new features like a
running member count and explicit owner type... it's cleaner and more
robust (eg, nodes know their owner, so they can casually switch between
list owners and remove themselves without the code knowing the owner).
This deprecates lws_dll, but since it's public it allows it to continue
to be built for 4.0 release if you give cmake LWS_WITH_DEPRECATED_LWS_DLL.
All remaining internal users of lws_dll are migrated to lws_dll2.
https://github.com/warmcat/libwebsockets/issues/1625
"dead bodies" that were sent but not processed by lws as server
will clog up and destroy transaction tracking if repeated POSTs
with keepalive are sent to nonexistant paths.
This patch introduces a DISCARD_BODY state that follows BODY
except the payload is not signalled to the protocol callback.
Calling transaction_completed() with pending body makes lws
enter DISCARD_BODY and retry transaction completed only after
the pending body is exhausted.
An lws context usually contains a processwide fd -> wsi lookup table.
This allows any possible fd returned by a *nix type OS to be immediately
converted to a wsi just by indexing an array of struct lws * the size of
the highest possible fd, as found by ulimit -n or similar.
This works modestly for Linux type systems where the default ulimit -n for
a process is 1024, it means a 4KB or 8KB lookup table for 32-bit or
64-bit systems.
However in the case your lws usage is much simpler, like one outgoing
client connection and no serving, this represents increasing waste. It's
made much worse if the system has a much larger default ulimit -n, eg 1M,
the table is occupying 4MB or 8MB, of which you will only use one.
Even so, because lws can't be sure the OS won't return a socket fd at any
number up to (ulimit -n - 1), it has to allocate the whole lookup table
at the moment.
This patch looks to see if the context creation info is setting
info->fd_limit_per_thread... if it leaves it at the default 0, then
everything is as it was before this patch. However if finds that
(info->fd_limit_per_thread * actual_number_of_service_threads) where
the default number of service threads is 1, is less than the fd limit
set by ulimit -n, lws switches to a slower lookup table scheme, which
only allocates the requested number of slots. Lookups happen then by
iterating the table and comparing rather than indexing the array
directly, which is obviously somewhat of a performance hit.
However in the case where you know lws will only have a very few wsi
maximum, this method can very usefully trade off speed to be able to
avoid the allocation sized by ulimit -n.
minimal examples for client that can make use of this are also modified
by this patch to use the smaller context allocations.
https://github.com/warmcat/libwebsockets/issues/1550
rx flow control needs to handle the situation that it is draining from
a previous rx flow control period, and the user code reasserts rx flow
control partway through that.
The accounting for the used rx then boils down to only trimming the
rxflow buflist we were "replaying" to consume however much we managed
to deliver of that this time before the rx flow control came again.
"Normal" rx consumption is wrong in this case, since we accounted for
it entirely in the rxflow cache buflist.
The patch recognizes this situation, does the accounting in the cache
buflist, and then lies to the caller that there was no rx consumption
to be accounted for at his level.
lws has been able to proxy h2 or h1 inbound connections to an
h1 onward connection for a while now. It's simple to use just
build with LWS_WITH_HTTP_PROXY and make a mount where the origin
is the onward connection details. Unix sockets can also be
used as the onward connection.
This patch extends the support to be able to also do the same for
inbound h2 or h1 ws upgrades to an h1 ws onward connection as well.
This allows you to offer completely different services in a
common URL space, including ones that connect back by ws / wss.
The callback flow is a bit more disruptive than doing the iteration
directly in your function. This helps by passing a user void *
into the callback set as an lws_dll[2]_foreach_safe() arg.
This allows the client stuff to understand that addresses beginning with '+'
represent unix sockets.
If the first character after the '+' is '@', it understands that the '@'
should be read as '\0', in order to use Linux "abstract namespace"
sockets.
Further the lws_parse_uri() helper is extended to understand the convention
that an address starting with + is a unix socket, and treats the socket
path as delimited by ':', eg
http://+/var/run/mysocket:/my/path
HTTP Proxy is updated to allow mounts to these unix socket paths.
Proxy connections go out on h1, but are dynamically translated to h1 or h2
on the incoming side.
Proxy usage of libhubbub is separated out... LWS_WITH_HTTP_PROXY is on by
default, and LWS_WITH_HUBBUB is off by default.
Various kinds of input stashing were replaced with a single buflist before
v3.0... this patch replaces the partial send arrangements with its own buflist
in the same way.
Buflists as the name says are growable lists of allocations in a linked-list
that take care of book-keeping what's added and removed (even if what is
removed is less than the current buffer on the list).
The immediate result is that we no longer have to freak out if we had a partial
buffered and new output is coming... we can just pile it on the end of the
buflist and keep draining the front of it.
Likewise we no longer need to be rabid about reporting multiple attempts to
send stuff without going back to the event loop, although not doing that
will introduce inefficiencies we don't have to term it "illegal" any more.
Since buflists have proven reliable on the input side and the logic for dealing
with truncated "non-network events" was already there this internal-only change
should be relatively self-contained.
HTTP server protocols have had for a while LWS_CALLBACK_HTTP_DROP/BIND_PROTOCOL
callbacks that mark when a wsi is attched to a protocol and detached.
It turns out this is generally useful for everything to know when a wsi is
joining a protocol and definitively completely finished with a protocol.
Particularly with client wsi where you provided the userdata externally, this
makes a clear point to free() it on the protocol binding being dropped.
This patch adds protocol bind / unbind callbacks to the role definition and
lets them operate on all roles. For the various roles
HTTP server: LWS_CALLBACK_HTTP_BIND/DROP_PROTOCOL as before
HTTP client: LWS_CALLBACK_CLIENT_HTTP_BIND/DROP_PROTOCOL
ws server: LWS_CALLBACK_WS_SERVER_BIND/DROP_PROTOCOL
ws client: LWS_CALLBACK_WS_CLIENT_BIND/DROP_PROTOCOL
raw file: LWS_CALLBACK_RAW_FILE_BIND/DROP_PROTOCOL
raw skt: LWS_CALLBACK_RAW_SKT_BIND/DROP_PROTOCOL
- split raw role into separate skt and file
- remove all special knowledge from the adoption
apis and migrate to core
- remove all special knowledge from client_connect
stuff, and have it discovered by iterating the
role callbacks to let those choose how to bind;
migrate to core
- retire the old deprecated client apis pre-
client_connect_info
Several new ops are planned for tls... so better to bite the bullet and
clean it out to the same level as roles + event-libs first.
Also adds a new travis target "mbedtls" and all the tests except
autobahn against mbedtls build.
You can build lws without support for ws, with -DLWS_ROLE_WS=0.
This is thanks to the role ops isolating all the ws-specific business
in the ws role.
Also retire more test apps replaced by minmal-examples.
This replaces the old test-app for echo with separate client and server
minimal versions.
The autobahn test script is made more autonomous and tests both
client and server.
1) Remove the whole ah rxbuf and put things on to the wsi buflist
This eliminates the whole detachability thing based on ah rxbuf
state... ah can always be detached.
2) Remove h2 scratch and put it on the wsi buflist
3) Remove preamble_rx and use the wsi buflist
This was used in the case adopted sockets had already been read.
Basically there are now only three forced service scenarios
- something in buflist (and not in state LRS_DEFERRING_ACTION)
- tls layer has buffered rx
- extension has buffered rx
This is a net removal of around 400 lines of special-casing.
This converts several of the selftests to return a status in their exit code
about whether they 'worked'.
A small bash script framework is added, with a selftest.sh in the mininmal
example dirs that support it, and a ./minimal-examples/selftests.sh script
that can be run from the build dir with no args that discovers and runs all
the selftest.sh scripts underneath.
That is also integrated into travis and the enabled tests must pass now for
travis to pass. Travis does not have a modern libuv so it can't run a
couple of tests which are nulled out if it sees it's running in travis env.
For h1 / ws, a combination of removing POLLIN wait and
stashing any unused rx lets us immediately respond to
rx flow control requests in a simple and effective way,
because the tcp socket is the stream.
But for muxed protocols like h2, that technique cannot
be used because we cannot silence the whole bundle of
streams because one can't handle any more rx dynamically.
There are control frames and content for other streams
serialized inbetween the flow controlled stream content.
We have no choice but to read to so we can see the other
things. Therefore for muxed protocols like h2 and spdy,
rx flow control boils down to tx credit manipulation
on individual streams to staunch the flow at the peer.
However that requires a round trip to take effect, any
transmitted packets that were in flight before the tx credit
reduction arrives at the remote peer are still going to come
and have to be dealt with by adding them to the stash.
This patch introduces lws_buflist scatter-gather type
buffer management for rxflow handling, so we can append
buffer segments in a linked-list to handle whatever rx
is unavoidably in flight on a stream that is trying to
assert rx flow control.
Since new roles may be incompatible with http, add support for
alpn names at the role struct, automatic generation of the
default list of alpn names that servers advertise, and the
ability to override the used alpn names per-vhost and per-
client connection.
This not only lets you modulate visibility or use of h2,
but also enables vhosts that only offer non-http roles,
as well as restricting http role vhosts to only alpn
identifiers related to http roles.
