1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-09 00:00:04 +01:00
Commit graph

472 commits

Author SHA1 Message Date
Andy Green
962e9ee345 wip: ss c++ classes
C++ APIs wrapping SS client


These are intended to provide an experimental protocol-independent c++
api even more abstracted than secure streams, along the lines of
"wget -Omyfile https://example.com/thing"

WIP
2020-12-02 10:15:50 +00:00
Andy Green
3952c2598d zip-fops: example: add concompressed mount too 2020-12-01 16:30:19 +00:00
Andy Green
2329dc6968 zip-fops: example: correct zipfile serving path 2020-12-01 15:38:20 +00:00
Andy Green
0ff5a1df75 ctest: sspc proxy minimal
CTest does not directly support daemon spawn as part of the test flow,
we have to specify it as a "fixture" dependency and then hack up daemonization
in a shellscript... this last part unfortunately limits its ability to run to
unix type platforms.

On those though, if the PROXY_API cmake option is enabled, the ctest flow will
spawn the proxy and run lws-minimal-secure-strems-client against it
2020-12-01 15:38:20 +00:00
Mykola Stryebkov
a03181301d smp: minimal client with several active connections 2020-11-28 10:58:38 +00:00
Andy Green
2bcae2b3b6 context: refactor destroy flow 2020-11-28 10:58:38 +00:00
Andy Green
9eb4c4fac2 client: rfc6724 dns results sorting
RFC6724 defines an ipv6-centric DNS result sorting algorithm, that
takes route and source address route information for the results
given by the DNS resolution, and sorts them in order of preferability,
which defines the order they should be tried in.

If LWS_WITH_NETLINK, then lws takes care about collecting and monitoring
the interface, route and source address information, and uses it to
perform the RFC6724 sorting to re-sort the DNS before trying to make
the connections.
2020-11-28 10:58:07 +00:00
Andy Green
1b7c0a08fc udp: convert to sa46
Let's stop using sockaddr and migrate to lws_sockaddr46 so the udp path
works more the same as the tcp / uds client connect path.
2020-11-26 09:23:30 +00:00
Andy Green
4e973c3cc5 ss: metadata: rename value to add reminder about heap 2020-10-19 16:34:54 +01:00
Andy Green
5f7da4a530 struct_json: adapt api test for 32-bit int on windows 2020-10-06 21:21:19 +01:00
Mykola Stryebkov
89f4b739e1 minimal: smp + foreign: libuv foreign loops
AG: various fixes and adaptation of test protocol to use ring lock
to protect global wsi list against modifications from other threads
while in use
2020-10-05 08:39:05 +01:00
Andy Green
ef8bfb2488 windows: build fixes 2020-09-30 06:42:09 +01:00
Andy Green
c75654c3f6 sequencer: disable by default 2020-09-29 09:29:59 +01:00
Andy Green
49e92ba089 http: add RFC7231 date and time helpers and retry-after handling
Teach lws how to deal with date: and retry-after:

Add quick selftest into apt-test-lws_tokenize

Expand lws_retry_sul_schedule_retry_wsi() to check for retry_after and
increase the backoff if a larger one found.

Finally, change SS h1 protocol to handle 503 + retry-after: as a
failure, and apply any increased backoff from retry-after
automatically.
2020-09-19 14:11:56 +01:00
Andy Green
33da902ed4 ss: policy: response code mapping
This adds a per-streamtype JSON mapping table in the policy.

In addition to the previous flow, it lets you generate custom
SS state notifications for specific http response codes, eg:

   "http_resp_map": [ { "530": 1530 }, { "531": 1531 } ],

It's not recommended to overload the transport-layer response
code with application layer responses.  It's better to return
a 200 and then in the application protocol inside http, explain
what happened from the application perspective, usually with
JSON.  But this is designed to let you handle existing systems
that do overload the transport layer response code.

SS states for user use start at LWSSSCS_USER_BASE, which is
1000.

You can do a basic test with minimal-secure-streams and --respmap
flag, this will go to httpbin.org and get a 404, and the warmcat.com
policy has the mapping for 404 -> LWSSSCS_USER_BASE (1000).

Since the mapping emits states, these are serialized and handled
like any other state in the proxy case.

The policy2c example / tool is also updated to handle the additional
mapping tables.
2020-09-16 13:10:26 +01:00
Andy Green
101b474217 ss: rx metadata
At the moment you can define and set per-stream metadata at the client,
which will be string-substituted and if configured in the policy, set in
related outgoing protocol specific content like h1 headers.

This patch extends the metadata concept to also check incoming protocol-
specific content like h1 headers and where it matches the binding in the
streamtype's metadata entry, make it available to the client by name, via
a new lws_ss_get_metadata() api.

Currently warmcat.com has additional headers for

server: lwsws                (well-known header name)
test-custom-header: hello    (custom header name)

minimal-secure-streams test is updated to try to recover these both
in direct and -client (via proxy) versions.  The corresponding metadata
part of the "mintest" stream policy from warmcat.com is

                        {
                                "srv": "server:"
                        }, {
                                "test": "test-custom-header:"
                        },

If built direct, or at the proxy, the stream has access to the static
policy metadata definitions and can store the rx metadata in the stream
metadata allocation, with heap-allocated a value.  For client side that
talks to a proxy, only the proxy knows the policy, and it returns rx
metadata inside the serialized link to the client, which stores it on
the heap attached to the stream.

In addition an optimization for mapping static policy metadata definitions
to individual stream handle metadata is changed to match by name.
2020-09-16 13:10:26 +01:00
Andy Green
d41bb16074 docs: force markdown to render correct path 2020-09-16 13:10:23 +01:00
Andy Green
40f7b84ff4 ws-client-binance: mbedtls wolfssl: trust ca cert explicitly 2020-08-31 16:51:37 +01:00
Andy Green
c6c7ab2b44 event libs: default to building as dynamically loaded plugins
Event lib support as it has been isn't scaling well, at the low level
libevent and libev headers have a namespace conflict so they can't
both be built into the same image, and at the distro level, binding
all the event libs to libwebsockets.so makes a bloaty situation for
packaging, lws will drag in all the event libs every time.

This patch implements the plan discussed here

https://github.com/warmcat/libwebsockets/issues/1980

and refactors the event lib support so they are built into isolated
plugins and bound at runtime according to what the application says
it wants to use.  The event lib plugins can be packaged individually
so that only the needed sets of support are installed (perhaps none
of them if the user code is OK with the default poll() loop).  And
dependent user code can mark the specific event loop plugin package
as required so pieces are added as needed.

The eventlib-foreign example is also refactored to build the selected
lib support isolated.

A readme is added detailing the changes and how to use them.

https://libwebsockets.org/git/libwebsockets/tree/READMEs/README.event-libs.md
2020-08-31 16:51:37 +01:00
Andy Green
d98101d1e3 plugins: generalize and provide public api
Move the common plugin scanning dir stuff to be based on lws_dir, which
already builds for windows.  Previously this was done via dirent for unix
and libuv for windows.

Reduce the dl plat stuff to just wrap instantiation and destruction of
dynlibs, establish common code in lib/misc/dir.c for plugin scanning
itself.

Migrate the libuv windows dl stuff to windows-plugins.c, so that he's
available even if later libuv loop support becomes and event lib plugin.

Remove the existing api exports scheme for plugins, just export a const struct
now which has a fixed header type but then whatever you want afterwards depending
on the class / purpose of the plugin.  Place a "class" string in the header so
there can be different kinds of plugins implying different types exported.

Make the plugin apis public and add support for filter by class string, and
per instantation / destruction callbacks so the subclassed header type can
do its thing for the plugin class.  The user provides a linked-list base
for his class of plugins, so he can manage them completely separately and
in user code / user export types.

Rip out some last hangers-on from generic sessions / tables.

This is all aimed at making the plugins support general enough so it can
provide event lib plugins later.
2020-08-31 16:51:37 +01:00
Andy Green
43311f3289 minimal-ws-client-binance
Add a minimal example showing how to write a binance client using
permessage_deflate and LCCSF_PRIORITIZE_READS to minimize latency.

This is partly informed by kutoga's example on
https://github.com/warmcat/libwebsockets/issues/2019 which was in
turn based on the existing ws client minimal example.
2020-08-31 16:51:37 +01:00
Andy Green
62c328244c cmake: fix things ignoring LWS_WITH_TLS 2020-08-31 16:51:36 +01:00
Andy Green
1ebe27e38f cmake: mark all the subprojects as explicitly C
We don't need a C++ compiler but if we don't spell it out, cmake
assumes it C + C++ compiler needed.
2020-08-31 16:51:36 +01:00
Andy Green
e8cbfea22d lws_smd: take care about build with disabled 2020-08-16 05:35:56 +01:00
Andy Green
44608abce6 sspc: fix intree build for sspc examples and increase post example body
Correct a comment about payload layout and add detailed comments about
dsh handling at proxy.

Increase the post size so it shows up fragmentation issues at the proxy.
2020-08-13 16:48:16 +01:00
Andy Green
dd3bae8c71 ss: multipart without processing
Change the default to not process multipart mime at SS layer.

If it's desired, then set "http_multipart_ss_in" true in the policy on the streamtype.

To test, use lws-minimal-secure-streams-avs, which uses SS processing as it is.

To check it without the processing, change #if 1 to #if 0 around the policy for
"http_multipart_ss_in" in both places in avs.c, and also enable the hexdump in ss_avs_metadata_rx()
also in avs.c, and observe the multipart framing is passed through unchanged.
2020-08-11 11:07:13 +01:00
Andy Green
03af613c13 sspc: minimal examples: improve PROXY_API detection to work outside tree 2020-08-10 15:04:10 +01:00
Andy Green
629c8138b1 sai: freebsd 2020-08-10 15:04:10 +01:00
Andy Green
4cd381f933 cmake: tls: wolfssl
PARENT_SCOPE needs adjusting in a few places for wolfssl to work, and
we need a second level export of USE_WOLFSSL through lib/CMakeLists.txt

Add noi/f32 Sai build for WOLFSSL + MINIMAL_EXAMPLES
2020-08-10 15:04:10 +01:00
Andy Green
a71cbe785e sspc: http POST: synthesize CONNECTED to provoke client body write 2020-08-10 15:04:10 +01:00
Andy Green
1b4bf38d5e sspc: add request_tx length variant
Add in the missing request_tx length variant, serialization and proxy
handling for it
2020-08-10 15:04:10 +01:00
Andy Green
5be8ff27d5 ss-server-raw
Add an example and some small changes for secure streams
serving raw data over a listening tcp socket
2020-07-28 09:21:45 +01:00
Andy Green
7eb36102a9 ss: server: h1, h2, ws basic support
Add initial support for defining servers using Secure Streams
policy and api semantics.

Serving h1, h2 and ws should be functional, the new minimal
example shows a combined http + SS server with an incrementing
ws message shown in the browser over tls, in around 200 lines
of user code.

NOP out anything to do with plugins, they're not currently used.

Update the docs correspondingly.
2020-07-27 12:05:24 +01:00
Soumendra Ganguly
beacbbb38c minimal-examples: http-server-dyn: Check if the first part of received url is our mountpoint
In the case of the http-server-dyn example, there is only one protocol.
The mounts will override the area of the url space they are set up for,
and the "gaps" will fall back to the default protocol, which in turn
defaults to protocol[0] at the moment.  So you always get the dyn
protocol action on unknown parts of the URL space.

This solves that behaviour by adding a new protocols[0] that calls
straight through to the default protocol handler lws_callback_http_dummy(),
and moves the dyn handler to be protocols[1].

In addition it shows how to check the url path in the dyn handler for
cases where user code doesn't want to add the second protocol.
2020-07-27 11:53:50 +01:00
Andy Green
625bade63e ss: static policy: dynamic vhost instantiation
Presently a vh is allocated per trust store at policy parsing-time, this
is no problem on a linux-class device or if you decide you need a dynamic
policy for functionality reasons.

However if you're in a constrained enough situation that the static policy
makes sense, in the case your trust stores do not have 100% duty cycle, ie,
are anyway always in use, the currently-unused vhosts and their x.509 stack
are sitting there taking up heap for no immediate benefit.

This patch modifies behaviour in ..._STATIC_POLICY_ONLY so that vhosts and
associated x.509 tls contexts are not instantiated until a secure stream using
them is created; they are refcounted, and when the last logical secure
stream using a vhost is destroyed, the vhost and its tls context is also
destroyed.

If another ss connection is created that wants to use the trust store, the
vhost and x.509 context is regenerated again as needed.

Currently the refcounting is by ss, it's also possible to move the refcounting
to be by connection.  The choice is between the delay to generate the vh
being visisble at logical ss creation-time, or at connection-time.  It's anyway
not preferable to have ss instantiated and taking up space with no associated
connection or connection attempt underway.

NB you will need to reprocess any static policies after this patch so they
conform to the trust_store changes.
2020-07-21 12:43:32 +01:00
Andy Green
da7ef0468b cgi: add spawn reap callback 2020-07-20 06:28:52 +01:00
Andy Green
6ae43ee06e lws_wsi_close: add helper 2020-07-20 06:28:52 +01:00
Sergey Radionov
fbadad7f52 glibc: fixed GSources destroy with non-default main context 2020-07-20 06:27:40 +01:00
Andy Green
886e93265a struct-lejp: handle no path match
For some patterns of JSON we return to parse at the outermost level and
meet a situation path_match is 0.  In some places we're looking at things
from perspective of path_match - 1... that does not seem to cause trouble on
x86_64 but can on aarch64, which is how it got noticed.

This logically protects those accesses by checking !!path_match.
2020-07-15 16:18:00 +01:00
Andy Green
abf9a7664f esp32: ctest: increase timeout for linkup and test to 60s 2020-07-15 16:18:00 +01:00
Andy Green
08b9a31e20 lws_json_simple: allow running into end
If the arg is unquoted, it's normal to run into the
end finding its extent.
2020-07-15 16:18:00 +01:00
Andy Green
77530f8ccb jwt: sign and create jws helper 2020-07-15 16:17:59 +01:00
Andy Green
1ae6ce37d3 lws_dll2: add helper for typed object name from owner list
There's a good pattern that's encouraged by using lws_struct pieces, that
we have an lws_dll2 owner with an array of objects listed in it that exist
in an lwsac.  And because it came from JSON, there is tending to be a
logical name for the objects.

This adds a typed helper and wrapper to scan the owner list looking for
a specific name (of a specified length, not NUL terminated) in a specific
member of the listed objects, which must be a NUL-terminated const char *.
Again this is a good pattern that's encouraged by use of lws_tokenize
to recover the name we're looking for.

So it leads to the helper that can cleanly search for a listed object of the
right name from an owner, and return the typed object pointer or NULL, from a
length-specified string.
2020-07-09 15:30:34 +01:00
Andy Green
4939b87e66 lws_netdev: use lws_settings
Perform the AP selection using lws_settings and a generic scan state machine
2020-07-07 15:23:19 +01:00
Andy Green
f902873634 ss: add timeout 2020-07-07 11:28:28 +01:00
Andy Green
dfed547529 glib: stop leaking idle and hrtimer source
This gives a valgrind result for lws-minimal-http-server-event-lib-foreign
that's clean for lws pieces, although glib is always dirty wrt thread local
storage and hashtable.
2020-07-07 11:28:28 +01:00
Andy Green
b72ab32c17 lws_netdev 2020-07-02 10:36:31 +01:00
Andy Green
63c1e8ba00 esp32-wrover-kit
Add lws_display and minimal example support for esp32-wrover to match wsp32-heltec-wb32

Since no usable buttons that don't affect something else on wrover kit, assumes
a button to 0V on GPIO14.
2020-06-30 19:35:41 +01:00
Andy Green
8eca7e17f2 lws_smd: system message distribution
- Add low level system message distibution framework
 - Add support for local Secure Streams to participate using _lws_smd streamtype
 - Add apit test and minimal example
 - Add SS proxy support for _lws_smd

See minimal-secure-streams-smd README.md
2020-06-27 07:57:22 +01:00
Andy Green
30f3606b0e context: option to disable system state management 2020-06-27 07:57:22 +01:00