Secure Streams is an optional layer on top of lws that separates policy
like endpoint selection and tls cert validation into a device JSON
policy document.
Code that wants to open a client connection just specifies a streamtype name,
and no longer deals with details like the endpoint, the protocol (!) or anything
else other than payloads and optionally generic metadata; the JSON policy
contains all the details for each streamtype. h1, h2, ws and mqtt client
connections are supported.
Logical secure streams outlive any particular connection and supports "nailed-up"
connectivity regardless of underlying connection stability.
Adds client support for MQTT QoS0 and QoS1, compatible with AWS IoT
Supports stream binding where independent client connections to the
same endpoint can mux on a single tcp + tls connection with topic
routing managed internally.
Headers related to ws or h2 are now elided if the ws or h2 role
is not enabled for build. In addition, a new build-time option
LWS_WITH_HTTP_UNCOMMON_HEADERS on by default allows removal of
less-common http headers to shrink the parser footprint.
Minilex is adapted to produce 8 different versions of the lex
table, chosen at build-time according to which headers are
included in the build.
If you don't need the unusual headers, or aren't using h2 or ws,
this chops down the size of the ah and the rodata needed to hold
the parsing table from 87 strings / pointers to 49, and the
parsing table from 1177 to 696 bytes.
This adds support for POST in both h1 and h2 queues / stream binding.
The previous queueing tried to keep the "leader" wsi who made the
actual connection around and have it act on the transaction queue
tail if it had done its own thing.
This refactors it so instead, who is the "leader" moves down the
queue and the queued guys inherit the fd, SSL * and queue from the
old leader as they take over.
This lets them operate in their own wsi identity directly and gets
rid of all the "effective wsi" checks, which was applied incompletely
and getting out of hand considering the separate lws_mux checks for
h2 and other muxed protocols alongside it.
This change also allows one wsi at a time to own the transaction for
POST. --post is added as an option to lws-minimal-http-client-multi
and 6 extra selftests with POST on h1/h2, pipelined or not and
staggered or not are added to the CI.
(Includes fixes from Yichen Gu)
Currently the incoming ebuf is always replaced to point to either a whole
buflist segment, or up to the (pt_serv_buf - LWS_PRE) length in the pt_serv_buf.
This is called on path for handling http read... some user code reasonably wants to
restrict the read size to what it can handle.
Change the other lws_buflist_aware_read() callers to zero ebuf before calling, and for
those have it keep the current behaviour; but if non-NULL ebuf.token on incoming, as
in http read path case, restrict both reported len of buflist content and the read length
to the incoming ebuf.len so the user code can control what it will get at one time.
Additionally muxed protocol wsi have no choice but to read what was sent to them
since it's HOL-blocking for other streams and its own WINDOW_UPDATEs. So add an
internal param to lws_buflist_aware_read() forcing read even if buflist content
is available.
There are some minor public api type improvements rather than cast everywhere
inside lws and user code to work around them... these changed from int to
size_t
- lws_buflist_use_segment() return
- lws_tokenize_t .len and .token_len
- lws_tokenize_cstr() length
- lws_get_peer_simple() namelen
- lws_get_peer_simple_fd() namelen, int fd -> lws_sockfd_type fd
- lws_write_numeric_address() len
- lws_sa46_write_numeric_address() len
These changes are typically a NOP for user code
This should be a NOP for h2 support and only affects internal
apis. But it lets us reuse the working and reliable h2 mux
arrangements directly in other protocols later, and share code
so building for h2 + new protocols can take advantage of common
mux child handling struct and code.
Break out common mux handling struct into its own type.
Convert all uses of members that used to be in wsi->h2 to wsi->mux
Audit all references to the members and break out generic helpers
for anything that is useful for other mux-capable protocols to
reuse wsi->mux related features.
On some platforms, it's possible that logging flow may reset errno. In the case where
we try to log errno on those platforms and afterwards try to query it, we will get a
nasty surprise that the logged errno is destroyed by the time we come to test it.
In the two cases of this in the tree at the moment, sample errno into a temp and
log and test the temp.
Thanks to Sakthi Ramabadran for finding this.
This teaches http client stuff how to handle 303 redirects... these
can happen after POST where the server side wants you to come back with
a GET to the Location: mentioned.
lws client will follow the redirect and force GET, this works for both
h1 and h2. Client protocol handler has to act differently if it finds
it is connecting for the initial POST or the subsequent GET, it can
find out which by checking a new api lws_http_is_redirected_to_get(wsi)
which returns nonzero if in GET mode.
Minimal example for server form-post has a new --303 switch to enable
this behaviour there and the client post example has additions to
check lws_http_is_redirected_to_get().
Resetting the ah and waiting a bit is the right strategy at the end of
http/1.1 client transaction. But it's wrong for h2... drop the ah
instead if it's the end of a client transaction on h2.
The %.*s is very handy to print strings where you have a length, but
there is no NUL termination. It's quite widely supported but at least
one vendor RTOS toolchain doesn't have it.
Since there aren't that many uses of it yet, audit all uses and
convert to a new helper lws_strnncpy() which uses the smaller of
two lengths.
lws has been able to generate client multipart mime as shown
in minimal-http-client-post, but it requires a lot of user
boilerplate to handle the boundary, related transaction header,
and multipart headers.
This patch adds a client creation flag to indicate it will
carry multipart mime, which autocreates the boundary string
and applies the transaction header with it, and an api to
form the boundary headers between the different mime parts
and the terminating boundary.
h1 and h2 has a bunch of code supporting autobinding outgoing client connections
to be streams in, or queued as pipelined on, the same / existing single network
connection, if it's to the same endpoint.
Adapt this http-specific code and active connection tracking to be usable for
generic muxable protocols the same way.
With http, the protocol doesn't indicate where the headers end and the
next transaction or body begin. Until now, we handled that for client
header response parsing by reading from the tls buffer bytewise.
This modernizes the code to read in up to 256-byte chunks and parse
the chunks in one hit (the parse API is already set up for doing this
elsewhere).
Now we have a generic input buflist, adapt the parser loop to go through
that and arrange that any leftovers are placed on there.
