1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-16 00:00:07 +01:00
Commit graph

84 commits

Author SHA1 Message Date
Andy Green
e502a5fb44 ss-h1: reset message state for repeated use
This lets one "opportunistic" mode http policy SS be able to be used
multiple times.
2021-09-14 05:31:29 +01:00
Andy Green
21697b3210 ss: handle DESTROY_ME from inside ss creation
Normally when doing a Client Connection Error handling,
we can action any ss relationship straight away since
we are in a wsi callback without any ss-aware parents
in the call stack.

But in the specific case we're doing the initial onward
wsi connection part on behalf of a ss, in fact the call
stack does have earlier  parents holding references on
the related ss.

For example

  secstream_h1 (ss-h1.c:470)                          CCE
  lws_inform_client_conn_fail (close.c:319)           fails early
  lws_client_connect_2_dnsreq (connect2.c:349)
  lws_http_client_connect_via_info2 (connect.c:71)
  lws_header_table_attach (parsers.c:291)
  rops_client_bind_h1 (ops-h1.c:1001)
  lws_client_connect_via_info (connect.c:429)         start onward connect
  _lws_ss_client_connect (secure-streams.c:859)
  _lws_ss_request_tx (secure-streams.c:1577)
  lws_ss_request_tx (secure-streams.c:1515)           request tx
  ss_cpd_state (captive-portal-detect.c:50)
  lws_ss_event_helper (secure-streams.c:408)
  lws_ss_create (secure-streams.c:1256)                SS Create

Under these conditions, we can't action the DESTROY_ME that
is coming when the CCE exhausts the retries.

This patch adds a flag that is set during the SS's onward wsi
connection attempt and causes it to stash rather than action
the result code.

The result code is brought out from the stash when we return to
_lws_ss_client_connect level, and passed up in the SS flow until
it is actioned, cleanly aborting the ss create.
2021-08-13 05:47:00 +01:00
Yichen Gu
5624659d37 conmon: only build if WITH_CONMON 2021-08-13 05:44:59 +01:00
Andy Green
c042ba8c65 mqtt: handle NULL mqtt publish metadata 2021-07-08 10:18:20 +01:00
Haopeng Li
526310dfea ss: h1: log dereferences NULL 2021-06-24 06:34:04 +01:00
Andy Green
e1ef230187 ss: h2: handle zero length COLON_PATH for metadata 2021-06-06 11:47:43 +01:00
Andy Green
da512b08c1 coverity: ss: handle NULL omd 2021-05-27 17:13:06 +01:00
Andy Green
fda7294bd0 ss: h2: leave the ss bound to its affiliated wsi until close clears it
In sai, on Xenial (only...) noticed that the wsi is still bound to the ss
handle, and can reference it even after the ss has been destroyed on
ss-testsfail sometimes.

Leave the handle knowing its wsi and able to detach it later during close.
2021-05-20 14:07:11 +01:00
Andy Green
afb77240ad h2: just log nwsi close and pass thru
User reports problems with the close / retry flow not happening if we don't
pass thru the nwsi close... it may be happening before the sid1 migration.

Just log it and don't end the handling before the passthru.  Logging it
because there was a reason for the change to not passing it through...
2021-04-21 19:32:03 +01:00
Chunho Lee
0276f3e635 ss: mqtt: Avoid CONNECTING to DISCONNECTED transition
Check previous states with 'ss_dangling_connected'.
If it did not visit CONNECTED, transit to UNREACHABLE
instead of DISCONNECTED.
2021-04-15 17:34:01 +01:00
Andy Green
402e97a679 ss: metrics: http: just report at wsi close
Defer recording the ss metrics histogram until wsi close, so it has a
chance to collect all the tags that apply.

Defer dumping metrics until the FINALIZE phase of context destroy, so we
had a chance to get any metrics recorded.
2021-04-08 06:47:54 +01:00
Haopeng
2a7ded310f ss: metrics: add result tag reflecting ss txn disposition
For http ss, add a tag result=SS_ACK_REMOTE or SS_NACK_REMOTE
2021-04-08 06:37:35 +01:00
Andy Green
d291c02a23 ss: sspc: add conmon performance telemetry
This provides a way to get ahold of LWS_WITH_CONMON telemetry from Secure
Streams, it works the same with direct onward connections or via the proxy.

You can mark streamtypes with a "perf": true policy attribute... this
causes the onward connections on those streamtypes to collect information
about the connection performance, and the unsorted DNS results.

Streams with that policy attribute receive extra data in their rx callback,
with the LWSSS_FLAG_PERF_JSON flag set on it, containing JSON describing the
performance of the onward connection taken from CONMON data, in a JSON
representation.  Streams without the "perf" attribute set never receive
this extra rx.

The received JSON is based on the CONMON struct info and looks like

{"peer":"46.105.127.147","dns_us":596,"sockconn_us":31382,"tls_us":28180,"txn_resp_us:23015,"dns":["2001:41d0:2:ee93::1","46.105.127.147"]}

A new minimal example minimal-secure-streams-perf is added that collects
this data on an HTTP GET from warmcat.com, and is built with a -client
version as well if LWS_WITH_SECURE_STREAMS_PROXY_API is set, that operates
via the ss proxy and produces the same result at the client.
2021-04-05 10:55:04 +01:00
Chunho Lee
25ae9facc9 mqtt: topic validation for different mqtt servers
AWS IoT enforces limits topic level and length. If 'aws_iot' is set
on the policy, the topic limits will be enforced for AWS IoT.
2021-03-30 07:38:37 +01:00
Sakthi Kannan
f3531ef673 mqtt: wildcard topic and topic to 256 chars
Adding supports to MQTT wildcard support, topic to 256 chars,
incorrect topic validation.
2021-03-30 07:38:37 +01:00
Sakthi Kannan
a088b72696 mqtt: Setting the CONNECTED state only when SUBACK is received
Setting the CONNECTED state only when SUBACK is received if the stream has
defined a subscription topic. This is to avoid SS from sending out SUBSCRIBE
right after CONNACK, even when the connection is not valid.
2021-03-30 07:38:32 +01:00
Andy Green
546c151555 ss: add set_metadata that takes a heap copy and make ss-h1 use it
Until now we set metadata value pointers into the onward wsi ah data
area... that's OK until we get a situation the wsi has gone away before we
have a chance to deliver the metadata over the proxy link.

Add a variant lws_ss_alloc_set_metadata() that allocates space on the heap
and takes a copy of the input metadata.  Change ss-h1 to alloc copies of
its metadata so we no longer race the wsi ah lifetime.
2021-03-14 15:05:18 +00:00
Andy Green
4804624905 ss: force check all set_metadata returns
lws_ss_set_metadata can fail... eg, due to transient OOM situation... if it does,
caller must take appropriate action like disconnect and retry.

So mark the api as requiring the result checking, and make sure all the
examples do it.
2021-03-14 08:58:26 +00:00
Andy Green
3f4623bb36 lws_metrics
There are a few build options that are trying to keep and report
various statistics

 - DETAILED_LATENCY
 - SERVER_STATUS
 - WITH_STATS

remove all those and establish a generic rplacement, lws_metrics.

lws_metrics makes its stats available via an lws_system ops function
pointer that the user code can set.

Openmetrics export is supported, for, eg, prometheus scraping.
2021-03-08 21:47:28 +00:00
Andy Green
f42c97baa9 route: on change also check source address still extant 2021-02-17 11:39:12 +00:00
Andy Green
c6d172c2ed ss: server mode fixes 2021-01-13 04:33:17 +00:00
Jed Lu
780a165a76 ss: http: mime parsing with unusual section boundaries 2021-01-11 08:12:46 +00:00
Andy Green
aa45de9e2a ss: enforce only valid state transitions
The various stream transitions for direct ss, SSPC, smd, and
different protocols are all handled in different code, let's
stop hoping for the best and add a state transition validation
function that is used everywhere we pass a state change to a
user callback, and knows what is valid for the user state()
callback to see next, given the last state it was shown.

Let's assert if lws manages to violate that so we can find
where the problem is and provide a stricter guarantee about
what user state handler will see, no matter if ss or sspc
or other cases.

To facilitate that, move the states to start from 1, where
0 indicates the state unset.
2021-01-11 08:12:43 +00:00
Jed Lu
c82910d30c ss: auth: sigv4
Add SS pieces for Sigv4 auth support
2021-01-05 10:56:38 +00:00
Andy Green
c9731c5f17 type comparisons: fixes
This is a huge patch that should be a global NOP.

For unix type platforms it enables -Wconversion to issue warnings (-> error)
for all automatic casts that seem less than ideal but are normally concealed
by the toolchain.

This is things like passing an int to a size_t argument.  Once enabled, I
went through all args on my default build (which build most things) and
tried to make the removed default cast explicit.

With that approach it neither change nor bloat the code, since it compiles
to whatever it was doing before, just with the casts made explicit... in a
few cases I changed some length args from int to size_t but largely left
the causes alone.

From now on, new code that is relying on less than ideal casting
will complain and nudge me to improve it by warnings.
2021-01-05 10:56:38 +00:00
Andy Green
e11d78dc5e sspc: cleanup bridged wsi 2021-01-04 05:40:12 +00:00
Andy Green
93f54c61c5 ss: enforce balanced CONNECT vs DISCONNECT 2021-01-04 05:26:50 +00:00
Andy Green
0ceba15d9c lws_lifecycle
This adds some new objects and helpers for keeping and logging
info on grouped allocations, a group is, eg, SS handles or client
wsis.

Allocated objects get a context-unique "tag" string intended to replace
%p / wsi pointers etc.  Pointers quickly become confusing when
allocations are freed and reused, the tag string won't repeat
until you produce 2^64 objects in a context.

In addition the tag string documents the object group, with prefixes
like "wsi-" or "vh-" and contain object-specific additional
information like the vhost name, address / port  or the role of the wsi.
At creation time the lws code can use a format string and args
to add whatever group-specific info makes sense, eg, a wsi bound
to a secure stream can also append the guid of the secure stream,
it's copied into the new object tag and so is still available
cleanly after the stream is destroyed if the wsi outlives it.
2021-01-04 05:26:50 +00:00
Andy Green
094bfc4e99 ss-hx: make sure hanging_some handled once 2020-12-27 08:41:58 +00:00
Andy Green
097bbbd1eb ss: client_connect and request_tx also return dispositions
Since client_connect and request_tx can be called from code that expects
the ss handle to be in scope, these calls can't deal with destroying the
ss handle and must pass the lws_ss_state_return_t disposition back to
the caller to handle.
2020-12-24 16:14:36 +00:00
Jed Lu
3c661ede9e ss: http: make sure we dont duplicate leading path slash if already present 2020-12-24 16:14:36 +00:00
Andy Green
68afe5b492 ss: minimal example: testsfail
A minimal example aimed at various SS failure paths.
2020-12-24 16:14:36 +00:00
Andy Green
36de3d8300 ss: h2: dont report nwsi close
SS protocol doesn't want to hear about the nwsi closing after
any streams on the protocol closed.
2020-12-24 16:14:36 +00:00
Andy Green
828b5cae24 ss: h2: set alpn
In order to force h2, we need to specify the client acceptable alpn as h2
2020-12-24 16:14:36 +00:00
Andy Green
e8420f31e8 ss: h1: also handle 429 same as 503 2020-12-24 16:14:36 +00:00
Jed Lu
e815ad2920 ss: handle nonzero content-length set by metadata correctly 2020-11-28 10:58:38 +00:00
Andy Green
426623433a ss-srv: destroy accepted clients on ws close 2020-11-28 10:58:38 +00:00
Andy Green
495a966302 mqtt: lws_system blobs for password username 2020-11-26 09:23:30 +00:00
Jed Lu
0f98972ab5 ss: skip null metadata 2020-11-26 09:23:30 +00:00
Andy Green
3216d4d087 ss: server: add foreach client cb api
Server SS maintains a list of accepted client ss, add an api allowing
iterating through the clients via a user callback.
2020-11-11 06:58:20 +00:00
Yichen Gu
3bfa00de19 ss: h2: ACK/NACK missing in H2 state flow 2020-11-02 12:03:16 +00:00
Andy Green
67d24d7c00 cleaning 2020-10-19 16:35:03 +01:00
Andy Green
4e973c3cc5 ss: metadata: rename value to add reminder about heap 2020-10-19 16:34:54 +01:00
Jed Lu
ee78b90c8c ss: take care to free any metadata heap values before overwrite
Break out the core ss_set_metadata action into a subfunction that
takes the lws_ss_metadata_t, and is fixed to retire heap-based
values before they go out of scope, and adapt the exported version
to call through to that.

Simplify extract_metadata() to reuse the subfunction as well, in
both well-known and custom header cases.
2020-10-07 19:20:12 +01:00
Andy Green
49e92ba089 http: add RFC7231 date and time helpers and retry-after handling
Teach lws how to deal with date: and retry-after:

Add quick selftest into apt-test-lws_tokenize

Expand lws_retry_sul_schedule_retry_wsi() to check for retry_after and
increase the backoff if a larger one found.

Finally, change SS h1 protocol to handle 503 + retry-after: as a
failure, and apply any increased backoff from retry-after
automatically.
2020-09-19 14:11:56 +01:00
Andy Green
33da902ed4 ss: policy: response code mapping
This adds a per-streamtype JSON mapping table in the policy.

In addition to the previous flow, it lets you generate custom
SS state notifications for specific http response codes, eg:

   "http_resp_map": [ { "530": 1530 }, { "531": 1531 } ],

It's not recommended to overload the transport-layer response
code with application layer responses.  It's better to return
a 200 and then in the application protocol inside http, explain
what happened from the application perspective, usually with
JSON.  But this is designed to let you handle existing systems
that do overload the transport layer response code.

SS states for user use start at LWSSSCS_USER_BASE, which is
1000.

You can do a basic test with minimal-secure-streams and --respmap
flag, this will go to httpbin.org and get a 404, and the warmcat.com
policy has the mapping for 404 -> LWSSSCS_USER_BASE (1000).

Since the mapping emits states, these are serialized and handled
like any other state in the proxy case.

The policy2c example / tool is also updated to handle the additional
mapping tables.
2020-09-16 13:10:26 +01:00
Andy Green
101b474217 ss: rx metadata
At the moment you can define and set per-stream metadata at the client,
which will be string-substituted and if configured in the policy, set in
related outgoing protocol specific content like h1 headers.

This patch extends the metadata concept to also check incoming protocol-
specific content like h1 headers and where it matches the binding in the
streamtype's metadata entry, make it available to the client by name, via
a new lws_ss_get_metadata() api.

Currently warmcat.com has additional headers for

server: lwsws                (well-known header name)
test-custom-header: hello    (custom header name)

minimal-secure-streams test is updated to try to recover these both
in direct and -client (via proxy) versions.  The corresponding metadata
part of the "mintest" stream policy from warmcat.com is

                        {
                                "srv": "server:"
                        }, {
                                "test": "test-custom-header:"
                        },

If built direct, or at the proxy, the stream has access to the static
policy metadata definitions and can store the rx metadata in the stream
metadata allocation, with heap-allocated a value.  For client side that
talks to a proxy, only the proxy knows the policy, and it returns rx
metadata inside the serialized link to the client, which stores it on
the heap attached to the stream.

In addition an optimization for mapping static policy metadata definitions
to individual stream handle metadata is changed to match by name.
2020-09-16 13:10:26 +01:00
Andy Green
4ae3ef51c1 ss: improve callback return consistency
Formalize the LWSSSSRET_ enums into a type "lws_ss_state_return_t"
returned by the rx, tx and state callbacks, and some private helpers
lws_ss_backoff() and lws_ss_event_helper().

Remove LWSSSSRET_SS_HANDLE_DESTROYED concept... the two helpers that could
have destroyed the ss and returned that, now return LWSSSSRET_DESTROY_ME
to the caller to perform or pass up to their caller instead.

Handle helper returns in all the ss protocols and update the rx / tx
calls to have their returns from rx / tx / event helper and ss backoff
all handled by unified code.
2020-08-31 16:51:37 +01:00
Andy Green
264c98c2c9 coverity: 62494: check system blob supposed to be used in h1 at runtime 2020-08-18 14:00:54 +01:00
Andy Green
80ea883796 coverity: 62550: back up assert with runtime NULL check 2020-08-18 14:00:54 +01:00