Mirrors/libwebsockets
1
0
Fork 0
mirror of https://github.com/warmcat/libwebsockets.git synced 2025-03-09 00:00:04 +01:00
Code Issues Releases Wiki Activity
libwebsockets/lib/roles/http/parsers.c

1727 lines
40 KiB
C
Raw Permalink Normal View History

break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
/*
* libwebsockets - small server side websockets and web server implementation
coding-style-cleanups.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-13 10:03:47 +00:00
*
license: switch LGPLv2.1+SLE parts to MIT
2019-08-14 10:44:14 +01:00
* Copyright (C) 2010 - 2019 Andy Green <andy@warmcat.com>
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
*
license: switch LGPLv2.1+SLE parts to MIT
2019-08-14 10:44:14 +01:00
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to
* deal in the Software without restriction, including without limitation the
* rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
* sell copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
*
license: switch LGPLv2.1+SLE parts to MIT
2019-08-14 10:44:14 +01:00
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
*
license: switch LGPLv2.1+SLE parts to MIT
2019-08-14 10:44:14 +01:00
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
*/
private.h: rename to contain dir Having unique private header names is a requirement of a particular platform build system it's desirable to work with
2019-08-15 10:49:52 +01:00
#include "private-lib-core.h"
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
static const unsigned char lextable_h1[] = {
private.h: rename to contain dir Having unique private header names is a requirement of a particular platform build system it's desirable to work with
2019-08-15 10:49:52 +01:00
#include "lextable.h"
add lexical parser for headers Profiling what happens during the ab test, one of the hotspots was strcasecmp in a loop looking for header name matches each time. This patch introduces a lexical parser that creates a state machine in 276 bytes that encodes all the known header names. The fsm is walked bytewise as chaacters come in... most states do not need any recursion to match or fail. The state machine output is cut-and-pasted into parsers.c as an unsigned char array. The fsm generator is a bit rough and ready, included in the tree but not built since normal mortals won't need to touch it. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-01-18 01:55:48 +08:00
};
upgrade and improve storage efficiency of minilex Until now minilex has done fine with providing a simple and fast header decode state machine. But for HTTP2.0, new headers must be added and it is already on the limit of table branching in 1 byte (already using +0xf8 of a max limit of 0xff). This changes the minilex format to improve storage size without loss of decode efficiency. It reduces the curent lws header table from 546 -> 403 bytes and upgrades the ability to increase table size by allowing jumps to increase from the old limit of +255 states to +65535 states, which should be enough for anything we ever want to do. The max number of terminals is also increased from 128 to 2048. Signed-off-by: Andy Green <andy.green@linaro.org>
2014-03-09 11:49:21 +08:00
#define FAIL_CHAR 0x08
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
#define UHO_NLEN 0
#define UHO_VLEN 2
#define UHO_LL 4
#define UHO_NAME 8
#endif
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
static struct allocated_headers *
_lws_create_ah(struct lws_context_per_thread *pt, ah_data_idx_t data_size)
{
struct allocated_headers *ah = lws_zalloc(sizeof(*ah), "ah struct");
if (!ah)
return NULL;
ah->data = lws_malloc(data_size, "ah data");
if (!ah->data) {
lws_free(ah);
return NULL;
}
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
ah->next = pt->http.ah_list;
pt->http.ah_list = ah;
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
ah->data_length = data_size;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pt->http.ah_pool_length++;
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
cleaning
2020-01-02 08:32:23 +00:00
lwsl_info("%s: created ah %p (size %d): pool length %u\n", __func__,
ah, (int)data_size, (unsigned int)pt->http.ah_pool_length);
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
return ah;
}
int
_lws_destroy_ah(struct lws_context_per_thread *pt, struct allocated_headers *ah)
{
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
lws_start_foreach_llp(struct allocated_headers **, a, pt->http.ah_list) {
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
if ((*a) == ah) {
*a = ah->next;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pt->http.ah_pool_length--;
cleaning
2020-01-02 08:32:23 +00:00
lwsl_info("%s: freed ah %p : pool length %u\n",
rtos: clean warnings where uint32_t is an unsigned long
2019-06-07 11:11:46 +01:00
__func__, ah,
cleaning
2020-01-02 08:32:23 +00:00
(unsigned int)pt->http.ah_pool_length);
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
if (ah->data)
lws_free(ah->data);
lws_free(ah);
return 0;
}
} lws_end_foreach_llp(a, next);
return 1;
}
client: fix redirects and allow ssl / non-ssl redirects
2017-02-09 15:25:01 +08:00
void
_lws_header_table_reset(struct allocated_headers *ah)
{
/* init the ah to reflect no headers or data have appeared yet */
memset(ah->frag_index, 0, sizeof(ah->frag_index));
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
memset(ah->frags, 0, sizeof(ah->frags));
client: fix redirects and allow ssl / non-ssl redirects
2017-02-09 15:25:01 +08:00
ah->nfrag = 0;
ah->pos = 0;
ah->http_response = 0;
LRS_DOING_TRANSACTION
2018-05-04 12:05:56 +08:00
ah->parser_state = WSI_TOKEN_NAME_PART;
ah->lextable_pos = 0;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
ah->unk_pos = 0;
ah: use unk_pos to clean up unknown header starts even without CUSTOM_HEADERS
2020-06-26 07:34:27 +01:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
ah->unk_ll_head = 0;
ah->unk_ll_tail = 0;
#endif
client: fix redirects and allow ssl / non-ssl redirects
2017-02-09 15:25:01 +08:00
}
lws_header_table_reset: make caller responsibility to clear down ah rx buffer There are two kinds of reaason to call lws_header_table_reset(), one is we are reallocating a destroyed ah to another wsi, and the other is we are moving to the next pipelined header set still on the same wsi, and we need a "weaker" reset that only clears down the state related to the header parsing, not everything about the ah context including the ah rx buffer. This patch moves the ah rxbuffer rxpos and rxlen resetting out of lws_header_table_reset() and to be the responsibility of the caller. Callers who are moving the ah to another wsi are patched to deal with resetting rxpos and rxlen and lws_http_transaction_completed() who only resets the ah when moving to the next pipelined headers, no longer wrongly clears the ah rxbuf. https://github.com/warmcat/libwebsockets/issues/638
2016-09-29 10:31:06 +08:00
// doesn't scrub the ah rxbuffer by default, parent must do if needed
ah http1.1 deal with pipelined headers properly Connections must hold an ah for the whole time they are processing one header set, even if eg, the headers are fragmented and it involves network roundtrip times. However on http1.1 / keepalive, it must drop the ah when there are no more header sets to deal with, and reacquire the ah later when more data appears. It's because the time between header sets / http1.1 requests is unbounded and the ah would be tied up forever. But in the case that we got pipelined http1.1 requests, even partial already buffered, we must keep the ah, resetting it instead of dropping it. Because we store the rx data conveniently in a per-tsi buffer since it only does one thing at a time per thread, we cannot go back to the event loop to await a new ah inside one service action. But no problem since we definitely already have an ah, let's just reuse it at http completion time if more rx is already buffered. NB: attack.sh makes request with echo | nc, this accidentally sends a trailing '\n' from the echo showing this problem. With this patch attack.sh can complete well. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-30 11:43:10 +08:00
void
smp: fixes
2018-03-02 14:22:49 +08:00
__lws_header_table_reset(struct lws *wsi, int autoservice)
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
{
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
struct allocated_headers *ah = wsi->http.ah;
adopt readbuf do service Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-27 11:42:22 +08:00
struct lws_context_per_thread *pt;
struct lws_pollfd *pfd;
ah owns rxbuf This is intended to solve a longstanding problem with the relationship between http/1.1 keep-alive and the service loop. Ah now contain an rx buffer which is used during header processing, and the ah may not be detached from the wsi until the rx buffer is exhausted. Having the rx buffer in the ah means we can delay using the rx until a later service loop. Ah which have pending rx force POLLIN service on the wsi they are attached to automatically, so we can interleave general service / connections with draining each ah rx buffer. The possible http/1.1 situations and their dispositions are: 1) exactly one set of http headers come. After processing, the ah is detached since no pending rx left. If more headers come later, a fresh ah is aqcuired when available and the rx flow control blocks the read until then. 2) more that one whole set of headers come and we remain in http mode (no upgrade). The ah is left attached and returns to the service loop after the first set of headers. We will get forced service due to the ah having pending content (respecting flowcontrol) and process the pending rx in the ah. If we use it all up, we will detach the ah. 3) one set of http headers come with ws traffic appended. We service the headers, do the upgrade, and keep the ah until the remaining ws content is used. When we exhausted the ws traffix in the ah rx buffer, we detach the ah. Since there can be any amount of http/1.1 pipelining on a connection, and each may be expensive to service, it's now enforced there is a return to the service loop after each header set is serviced on a connection. When I added the forced service for ah with pending buffering, I added support for it to the windows plat code. However this is untested. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-15 12:37:04 +08:00
/* if we have the idea we're resetting 'our' ah, must be bound to one */
assert(ah);
/* ah also concurs with ownership */
assert(ah->wsi == wsi);
lws_reset_header_table dont reset NULL This is only a temporary, fragile solution. https://github.com/warmcat/libwebsockets/issues/416 Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-09 09:15:02 +08:00
client: fix redirects and allow ssl / non-ssl redirects
2017-02-09 15:25:01 +08:00
_lws_header_table_reset(ah);
ah owns rxbuf This is intended to solve a longstanding problem with the relationship between http/1.1 keep-alive and the service loop. Ah now contain an rx buffer which is used during header processing, and the ah may not be detached from the wsi until the rx buffer is exhausted. Having the rx buffer in the ah means we can delay using the rx until a later service loop. Ah which have pending rx force POLLIN service on the wsi they are attached to automatically, so we can interleave general service / connections with draining each ah rx buffer. The possible http/1.1 situations and their dispositions are: 1) exactly one set of http headers come. After processing, the ah is detached since no pending rx left. If more headers come later, a fresh ah is aqcuired when available and the rx flow control blocks the read until then. 2) more that one whole set of headers come and we remain in http mode (no upgrade). The ah is left attached and returns to the service loop after the first set of headers. We will get forced service due to the ah having pending content (respecting flowcontrol) and process the pending rx in the ah. If we use it all up, we will detach the ah. 3) one set of http headers come with ws traffic appended. We service the headers, do the upgrade, and keep the ah until the remaining ws content is used. When we exhausted the ws traffix in the ah rx buffer, we detach the ah. Since there can be any amount of http/1.1 pipelining on a connection, and each may be expensive to service, it's now enforced there is a return to the service loop after each header set is serviced on a connection. When I added the forced service for ah with pending buffering, I added support for it to the windows plat code. However this is untested. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-15 12:37:04 +08:00
/* since we will restart the ah, our new headers are not completed */
ah reset: enforce reset of parsing_complete
2017-06-28 09:55:34 +08:00
wsi->hdr_parsing_completed = 0;
adopt variant with preamble rx Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-24 11:05:56 +08:00
ah: allow configurable ah hold timeout
2017-07-26 11:49:41 +08:00
/* while we hold the ah, keep a timeout on the wsi */
smp: fixes
2018-03-02 14:22:49 +08:00
__lws_set_timeout(wsi, PENDING_TIMEOUT_HOLDING_AH,
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
wsi->a.vhost->timeout_secs_ah_idle);
ah: allow configurable ah hold timeout
2017-07-26 11:49:41 +08:00
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
time(&ah->assigned);
windows: cleanup wrong and duplicated socket validity helpers https://github.com/warmcat/libwebsockets/issues/1259
2018-05-06 07:19:21 +08:00
if (wsi->position_in_fds_table != LWS_NO_FDS_POS &&
lws_buflist_next_segment_len(&wsi->buflist, NULL) &&
refactor: absorb other caches into buflist 1) Remove the whole ah rxbuf and put things on to the wsi buflist This eliminates the whole detachability thing based on ah rxbuf state... ah can always be detached. 2) Remove h2 scratch and put it on the wsi buflist 3) Remove preamble_rx and use the wsi buflist This was used in the case adopted sockets had already been read. Basically there are now only three forced service scenarios - something in buflist (and not in state LRS_DEFERRING_ACTION) - tls layer has buffered rx - extension has buffered rx This is a net removal of around 400 lines of special-casing.
2018-04-17 15:35:15 +08:00
autoservice) {
lwsl_debug("%s: service on readbuf ah\n", __func__);
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
pt = &wsi->a.context->pt[(int)wsi->tsi];
refactor: absorb other caches into buflist 1) Remove the whole ah rxbuf and put things on to the wsi buflist This eliminates the whole detachability thing based on ah rxbuf state... ah can always be detached. 2) Remove h2 scratch and put it on the wsi buflist 3) Remove preamble_rx and use the wsi buflist This was used in the case adopted sockets had already been read. Basically there are now only three forced service scenarios - something in buflist (and not in state LRS_DEFERRING_ACTION) - tls layer has buffered rx - extension has buffered rx This is a net removal of around 400 lines of special-casing.
2018-04-17 15:35:15 +08:00
/*
* Unlike a normal connect, we have the headers already
* (or the first part of them anyway)
*/
pfd = &pt->fds[wsi->position_in_fds_table];
pfd->revents |= LWS_POLLIN;
lwsl_err("%s: calling service\n", __func__);
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
lws_service_fd_tsi(wsi->a.context, pfd, wsi->tsi);
adopt variant with preamble rx Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-24 11:05:56 +08:00
}
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
}
smp: fixes
2018-03-02 14:22:49 +08:00
void
lws_header_table_reset(struct lws *wsi, int autoservice)
{
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
struct lws_context_per_thread *pt = &wsi->a.context->pt[(int)wsi->tsi];
smp: fixes
2018-03-02 14:22:49 +08:00
lws_pt_lock(pt, __func__);
__lws_header_table_reset(wsi, autoservice);
lws_pt_unlock(pt);
}
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
static void
_lws_header_ensure_we_are_on_waiting_list(struct lws *wsi)
{
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
struct lws_context_per_thread *pt = &wsi->a.context->pt[(int)wsi->tsi];
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
struct lws_pollargs pa;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
struct lws **pwsi = &pt->http.ah_wait_list;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
while (*pwsi) {
if (*pwsi == wsi)
return;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pwsi = &(*pwsi)->http.ah_wait_list;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
}
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_info("%s: wsi: %s\n", __func__, lws_wsi_tag(wsi));
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah_wait_list = pt->http.ah_wait_list;
pt->http.ah_wait_list = wsi;
pt->http.ah_wait_list_length++;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
/* we cannot accept input then */
_lws_change_pollfd(wsi, LWS_POLLIN, 0, &pa);
}
static int
__lws_remove_from_ah_waiting_list(struct lws *wsi)
{
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
struct lws_context_per_thread *pt = &wsi->a.context->pt[(int)wsi->tsi];
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
struct lws **pwsi =&pt->http.ah_wait_list;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
while (*pwsi) {
if (*pwsi == wsi) {
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_info("%s: wsi %s\n", __func__, lws_wsi_tag(wsi));
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
/* point prev guy to our next */
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
*pwsi = wsi->http.ah_wait_list;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
/* we shouldn't point anywhere now */
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah_wait_list = NULL;
pt->http.ah_wait_list_length--;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
return 1;
}
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pwsi = &(*pwsi)->http.ah_wait_list;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
}
return 0;
}
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
int LWS_WARN_UNUSED_RESULT
adopt readbuf do service Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-27 11:42:22 +08:00
lws_header_table_attach(struct lws *wsi, int autoservice)
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
{
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
struct lws_context *context = wsi->a.context;
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
struct lws_context_per_thread *pt = &context->pt[(int)wsi->tsi];
struct lws_pollargs pa;
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
int n;
client: MQTT Adds client support for MQTT QoS0 and QoS1, compatible with AWS IoT Supports stream binding where independent client connections to the same endpoint can mux on a single tcp + tls connection with topic routing managed internally.
2020-02-25 13:28:25 +00:00
#if defined(LWS_ROLE_MQTT) && defined(LWS_WITH_CLIENT)
if (lwsi_role_mqtt(wsi))
goto connect_via_info2;
#endif
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_info("%s: %s: ah %p (tsi %d, count = %d) in\n", __func__,
lws_wsi_tag(wsi), (void *)wsi->http.ah, wsi->tsi,
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pt->http.ah_count_in_use);
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
role: raw-proxy
2018-11-29 08:29:48 +08:00
if (!lwsi_role_http(wsi)) {
lwsl_err("%s: bad role %s\n", __func__, wsi->role_ops->name);
assert(0);
return -1;
}
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
lws_pt_lock(pt, __func__);
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
/* if we are already bound to one, just clear it down */
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
if (wsi->http.ah) {
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
lwsl_info("%s: cleardown\n", __func__);
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
goto reset;
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
}
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
n = pt->http.ah_count_in_use == (int)context->max_http_header_pool;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
#if defined(LWS_WITH_PEER_LIMITS)
lws_metrics There are a few build options that are trying to keep and report various statistics - DETAILED_LATENCY - SERVER_STATUS - WITH_STATS remove all those and establish a generic rplacement, lws_metrics. lws_metrics makes its stats available via an lws_system ops function pointer that the user code can set. Openmetrics export is supported, for, eg, prometheus scraping.
2021-01-06 15:08:22 +00:00
if (!n)
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
n = lws_peer_confirm_ah_attach_ok(context, wsi->peer);
#endif
if (n) {
/*
* Pool is either all busy, or we don't want to give this
* particular guy an ah right now...
*
* Make sure we are on the waiting list, and return that we
* weren't able to provide the ah
*/
_lws_header_ensure_we_are_on_waiting_list(wsi);
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
goto bail;
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
}
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
__lws_remove_from_ah_waiting_list(wsi);
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah = _lws_create_ah(pt, context->max_http_header_data);
if (!wsi->http.ah) { /* we could not create an ah */
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
_lws_header_ensure_we_are_on_waiting_list(wsi);
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
goto bail;
}
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah->in_use = 1;
wsi->http.ah->wsi = wsi; /* mark our owner */
pt->http.ah_count_in_use++;
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
clean: general whitespace cleanup
2018-11-23 08:47:56 +08:00
#if defined(LWS_WITH_PEER_LIMITS) && (defined(LWS_ROLE_H1) || \
defined(LWS_ROLE_H2))
lws_mutex_refcount This creates a "pthread mutex with a reference count" using gcc / clang atomic intrinsics + pthreads. Both pt and context locks are moved to use this, pt already had reference counting but it's new for context.
2018-06-27 07:15:39 +08:00
lws_context_lock(context, "ah attach"); /* <========================= */
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
if (wsi->peer)
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->peer->http.count_ah++;
lws_context_unlock(context); /* ====================================> */
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
#endif
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
_lws_change_pollfd(wsi, 0, LWS_POLLIN, &pa);
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_info("%s: did attach wsi %s: ah %p: count %d (on exit)\n", __func__,
lws_wsi_tag(wsi), (void *)wsi->http.ah, pt->http.ah_count_in_use);
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
reset:
smp: fixes
2018-03-02 14:22:49 +08:00
__lws_header_table_reset(wsi, autoservice);
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
lws_pt_unlock(pt);
client: secure streams Secure Streams is an optional layer on top of lws that separates policy like endpoint selection and tls cert validation into a device JSON policy document. Code that wants to open a client connection just specifies a streamtype name, and no longer deals with details like the endpoint, the protocol (!) or anything else other than payloads and optionally generic metadata; the JSON policy contains all the details for each streamtype. h1, h2, ws and mqtt client connections are supported. Logical secure streams outlive any particular connection and supports "nailed-up" connectivity regardless of underlying connection stability.
2020-02-29 12:37:24 +00:00
#if defined(LWS_WITH_CLIENT)
client: MQTT Adds client support for MQTT QoS0 and QoS1, compatible with AWS IoT Supports stream binding where independent client connections to the same endpoint can mux on a single tcp + tls connection with topic routing managed internally.
2020-02-25 13:28:25 +00:00
#if defined(LWS_ROLE_MQTT)
connect_via_info2:
#endif
refactor mode and states into roles and states
2018-04-02 11:55:17 +08:00
if (lwsi_role_client(wsi) && lwsi_state(wsi) == LRS_UNCONNECTED)
refactor: split out adoption and client apis to core - split raw role into separate skt and file - remove all special knowledge from the adoption apis and migrate to core - remove all special knowledge from client_connect stuff, and have it discovered by iterating the role callbacks to let those choose how to bind; migrate to core - retire the old deprecated client apis pre- client_connect_info
2018-05-10 16:13:26 +08:00
if (!lws_http_client_connect_via_info2(wsi))
client handle connection fail at second phase properly Signed-off-by: Andy Green <andy@warmcat.com>
2016-05-08 16:58:18 +08:00
/* our client connect has failed, the wsi
* has been closed
*/
return -1;
client accept connection request even if no free ah It can join the free ah list and pick up client connect processing later when the ah becomes available; this simplifies the code doing the request since he won't have to deal with unexpected failures / retries based on dynamic ah availability. To do this though we have to handle that the connect_info members may not have scope that lets them still exist after we return from the first connect call, we stash them in a malloc'd buffer so the connect processing can have them much later even so. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-29 14:19:16 +08:00
#endif
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
return 0;
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
bail:
lws_pt_unlock(pt);
return 1;
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
}
pthreads: more locking
2018-03-05 16:49:28 +08:00
int __lws_header_table_detach(struct lws *wsi, int autoservice)
Adding header table free function to make sure we free cleanly everywhere.
2014-11-07 11:20:59 +08:00
{
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
struct lws_context *context = wsi->a.context;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
struct allocated_headers *ah = wsi->http.ah;
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
struct lws_context_per_thread *pt = &context->pt[(int)wsi->tsi];
struct lws_pollargs pa;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
struct lws **pwsi, **pwsi_eligible;
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
time_t now;
ah: enforce waiting list detach
2017-07-19 04:06:15 +08:00
__lws_remove_from_ah_waiting_list(wsi);
ah: require parsing complete before detach Introduce helpers to force to detachable state and to test the ah is in a detachable state. Require not only the ah rx buffer is all used, but that the wsi has completed a full set of headers.
2017-06-28 12:13:13 +08:00
if (!ah)
return 0;
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_info("%s: %s: ah %p (tsi=%d, count = %d)\n", __func__,
lws_wsi_tag(wsi), (void *)ah, wsi->tsi,
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pt->http.ah_count_in_use);
ah owns rxbuf This is intended to solve a longstanding problem with the relationship between http/1.1 keep-alive and the service loop. Ah now contain an rx buffer which is used during header processing, and the ah may not be detached from the wsi until the rx buffer is exhausted. Having the rx buffer in the ah means we can delay using the rx until a later service loop. Ah which have pending rx force POLLIN service on the wsi they are attached to automatically, so we can interleave general service / connections with draining each ah rx buffer. The possible http/1.1 situations and their dispositions are: 1) exactly one set of http headers come. After processing, the ah is detached since no pending rx left. If more headers come later, a fresh ah is aqcuired when available and the rx flow control blocks the read until then. 2) more that one whole set of headers come and we remain in http mode (no upgrade). The ah is left attached and returns to the service loop after the first set of headers. We will get forced service due to the ah having pending content (respecting flowcontrol) and process the pending rx in the ah. If we use it all up, we will detach the ah. 3) one set of http headers come with ws traffic appended. We service the headers, do the upgrade, and keep the ah until the remaining ws content is used. When we exhausted the ws traffix in the ah rx buffer, we detach the ah. Since there can be any amount of http/1.1 pipelining on a connection, and each may be expensive to service, it's now enforced there is a return to the service loop after each header set is serviced on a connection. When I added the forced service for ah with pending buffering, I added support for it to the windows plat code. However this is untested. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-15 12:37:04 +08:00
improve timeout and ah list comments Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-25 21:39:01 +08:00
/* we did have an ah attached */
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
time(&now);
ah detach: don't print held time if actually detached Confusingly an ah held time was printed for a previously detached ah. Clear down the time field when detaching the ah so this can't happen any more.
2016-07-19 09:38:48 +08:00
if (ah->assigned && now - ah->assigned > 3) {
ah pool lifetime use dynamic rxpos Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-19 11:47:52 +08:00
/*
* we're detaching the ah, but it was held an
* unreasonably long time
*/
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_debug("%s: %s: ah held %ds, role/state 0x%lx 0x%x,"
"\n", __func__, lws_wsi_tag(wsi),
(int)(now - ah->assigned),
rtos: clean warnings where uint32_t is an unsigned long
2019-06-07 11:11:46 +01:00
(unsigned long)lwsi_role(wsi), lwsi_state(wsi));
ah pool lifetime use dynamic rxpos Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-19 11:47:52 +08:00
}
ah owns rxbuf This is intended to solve a longstanding problem with the relationship between http/1.1 keep-alive and the service loop. Ah now contain an rx buffer which is used during header processing, and the ah may not be detached from the wsi until the rx buffer is exhausted. Having the rx buffer in the ah means we can delay using the rx until a later service loop. Ah which have pending rx force POLLIN service on the wsi they are attached to automatically, so we can interleave general service / connections with draining each ah rx buffer. The possible http/1.1 situations and their dispositions are: 1) exactly one set of http headers come. After processing, the ah is detached since no pending rx left. If more headers come later, a fresh ah is aqcuired when available and the rx flow control blocks the read until then. 2) more that one whole set of headers come and we remain in http mode (no upgrade). The ah is left attached and returns to the service loop after the first set of headers. We will get forced service due to the ah having pending content (respecting flowcontrol) and process the pending rx in the ah. If we use it all up, we will detach the ah. 3) one set of http headers come with ws traffic appended. We service the headers, do the upgrade, and keep the ah until the remaining ws content is used. When we exhausted the ws traffix in the ah rx buffer, we detach the ah. Since there can be any amount of http/1.1 pipelining on a connection, and each may be expensive to service, it's now enforced there is a return to the service loop after each header set is serviced on a connection. When I added the forced service for ah with pending buffering, I added support for it to the windows plat code. However this is untested. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-15 12:37:04 +08:00
ah detach: don't print held time if actually detached Confusingly an ah held time was printed for a previously detached ah. Clear down the time field when detaching the ah so this can't happen any more.
2016-07-19 09:38:48 +08:00
ah->assigned = 0;
ah owns rxbuf This is intended to solve a longstanding problem with the relationship between http/1.1 keep-alive and the service loop. Ah now contain an rx buffer which is used during header processing, and the ah may not be detached from the wsi until the rx buffer is exhausted. Having the rx buffer in the ah means we can delay using the rx until a later service loop. Ah which have pending rx force POLLIN service on the wsi they are attached to automatically, so we can interleave general service / connections with draining each ah rx buffer. The possible http/1.1 situations and their dispositions are: 1) exactly one set of http headers come. After processing, the ah is detached since no pending rx left. If more headers come later, a fresh ah is aqcuired when available and the rx flow control blocks the read until then. 2) more that one whole set of headers come and we remain in http mode (no upgrade). The ah is left attached and returns to the service loop after the first set of headers. We will get forced service due to the ah having pending content (respecting flowcontrol) and process the pending rx in the ah. If we use it all up, we will detach the ah. 3) one set of http headers come with ws traffic appended. We service the headers, do the upgrade, and keep the ah until the remaining ws content is used. When we exhausted the ws traffix in the ah rx buffer, we detach the ah. Since there can be any amount of http/1.1 pipelining on a connection, and each may be expensive to service, it's now enforced there is a return to the service loop after each header set is serviced on a connection. When I added the forced service for ah with pending buffering, I added support for it to the windows plat code. However this is untested. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-15 12:37:04 +08:00
/* if we think we're detaching one, there should be one in use */
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
assert(pt->http.ah_count_in_use > 0);
ah owns rxbuf This is intended to solve a longstanding problem with the relationship between http/1.1 keep-alive and the service loop. Ah now contain an rx buffer which is used during header processing, and the ah may not be detached from the wsi until the rx buffer is exhausted. Having the rx buffer in the ah means we can delay using the rx until a later service loop. Ah which have pending rx force POLLIN service on the wsi they are attached to automatically, so we can interleave general service / connections with draining each ah rx buffer. The possible http/1.1 situations and their dispositions are: 1) exactly one set of http headers come. After processing, the ah is detached since no pending rx left. If more headers come later, a fresh ah is aqcuired when available and the rx flow control blocks the read until then. 2) more that one whole set of headers come and we remain in http mode (no upgrade). The ah is left attached and returns to the service loop after the first set of headers. We will get forced service due to the ah having pending content (respecting flowcontrol) and process the pending rx in the ah. If we use it all up, we will detach the ah. 3) one set of http headers come with ws traffic appended. We service the headers, do the upgrade, and keep the ah until the remaining ws content is used. When we exhausted the ws traffix in the ah rx buffer, we detach the ah. Since there can be any amount of http/1.1 pipelining on a connection, and each may be expensive to service, it's now enforced there is a return to the service loop after each header set is serviced on a connection. When I added the forced service for ah with pending buffering, I added support for it to the windows plat code. However this is untested. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-15 12:37:04 +08:00
/* and this specific one should have been in use */
ah detach: don't print held time if actually detached Confusingly an ah held time was printed for a previously detached ah. Clear down the time field when detaching the ah so this can't happen any more.
2016-07-19 09:38:48 +08:00
assert(ah->in_use);
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
memset(&wsi->http.ah, 0, sizeof(wsi->http.ah));
smp: h2 children must go on same tsi as parent We can't have a situation where stream wsis inside an h2 connection are serviced by different threads than the actual parent connection.
2018-04-27 12:49:42 +08:00
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
#if defined(LWS_WITH_PEER_LIMITS)
smp: h2 children must go on same tsi as parent We can't have a situation where stream wsis inside an h2 connection are serviced by different threads than the actual parent connection.
2018-04-27 12:49:42 +08:00
if (ah->wsi)
lws_peer_track_ah_detach(context, wsi->peer);
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
#endif
smp: h2 children must go on same tsi as parent We can't have a situation where stream wsis inside an h2 connection are serviced by different threads than the actual parent connection.
2018-04-27 12:49:42 +08:00
ah->wsi = NULL; /* no owner */
client: secure streams Secure Streams is an optional layer on top of lws that separates policy like endpoint selection and tls cert validation into a device JSON policy document. Code that wants to open a client connection just specifies a streamtype name, and no longer deals with details like the endpoint, the protocol (!) or anything else other than payloads and optionally generic metadata; the JSON policy contains all the details for each streamtype. h1, h2, ws and mqtt client connections are supported. Logical secure streams outlive any particular connection and supports "nailed-up" connectivity regardless of underlying connection stability.
2020-02-29 12:37:24 +00:00
wsi->http.ah = NULL;
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pwsi = &pt->http.ah_wait_list;
ah: enforce waiting list detach
2017-07-19 04:06:15 +08:00
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
/* oh there is nobody on the waiting list... leave the ah unattached */
if (!*pwsi)
goto nobody_usable_waiting;
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
/*
* at least one wsi on the same tsi is waiting, give it to oldest guy
* who is allowed to take it (if any)
*/
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_info("%s: pt wait list %s\n", __func__, lws_wsi_tag(*pwsi));
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
wsi = NULL;
pwsi_eligible = NULL;
while (*pwsi) {
#if defined(LWS_WITH_PEER_LIMITS)
/* are we willing to give this guy an ah? */
if (!lws_peer_confirm_ah_attach_ok(context, (*pwsi)->peer))
#endif
{
wsi = *pwsi;
pwsi_eligible = pwsi;
}
lws_metrics There are a few build options that are trying to keep and report various statistics - DETAILED_LATENCY - SERVER_STATUS - WITH_STATS remove all those and establish a generic rplacement, lws_metrics. lws_metrics makes its stats available via an lws_system ops function pointer that the user code can set. Openmetrics export is supported, for, eg, prometheus scraping.
2021-01-06 15:08:22 +00:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pwsi = &(*pwsi)->http.ah_wait_list;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
}
if (!wsi) /* everybody waiting already has too many ah... */
goto nobody_usable_waiting;
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
clean: general whitespace cleanup
2018-11-23 08:47:56 +08:00
lwsl_info("%s: transferring ah to last eligible wsi in wait list "
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
"%s (wsistate 0x%lx)\n", __func__, lws_wsi_tag(wsi),
rtos: clean warnings where uint32_t is an unsigned long
2019-06-07 11:11:46 +01:00
(unsigned long)wsi->wsistate);
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah = ah;
ah owns rxbuf This is intended to solve a longstanding problem with the relationship between http/1.1 keep-alive and the service loop. Ah now contain an rx buffer which is used during header processing, and the ah may not be detached from the wsi until the rx buffer is exhausted. Having the rx buffer in the ah means we can delay using the rx until a later service loop. Ah which have pending rx force POLLIN service on the wsi they are attached to automatically, so we can interleave general service / connections with draining each ah rx buffer. The possible http/1.1 situations and their dispositions are: 1) exactly one set of http headers come. After processing, the ah is detached since no pending rx left. If more headers come later, a fresh ah is aqcuired when available and the rx flow control blocks the read until then. 2) more that one whole set of headers come and we remain in http mode (no upgrade). The ah is left attached and returns to the service loop after the first set of headers. We will get forced service due to the ah having pending content (respecting flowcontrol) and process the pending rx in the ah. If we use it all up, we will detach the ah. 3) one set of http headers come with ws traffic appended. We service the headers, do the upgrade, and keep the ah until the remaining ws content is used. When we exhausted the ws traffix in the ah rx buffer, we detach the ah. Since there can be any amount of http/1.1 pipelining on a connection, and each may be expensive to service, it's now enforced there is a return to the service loop after each header set is serviced on a connection. When I added the forced service for ah with pending buffering, I added support for it to the windows plat code. However this is untested. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-15 12:37:04 +08:00
ah->wsi = wsi; /* new owner */
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
smp: fixes
2018-03-02 14:22:49 +08:00
__lws_header_table_reset(wsi, autoservice);
clean: general whitespace cleanup
2018-11-23 08:47:56 +08:00
#if defined(LWS_WITH_PEER_LIMITS) && (defined(LWS_ROLE_H1) || \
defined(LWS_ROLE_H2))
lws_mutex_refcount This creates a "pthread mutex with a reference count" using gcc / clang atomic intrinsics + pthreads. Both pt and context locks are moved to use this, pt already had reference counting but it's new for context.
2018-06-27 07:15:39 +08:00
lws_context_lock(context, "ah detach"); /* <========================= */
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
if (wsi->peer)
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->peer->http.count_ah++;
smp: h2 children must go on same tsi as parent We can't have a situation where stream wsis inside an h2 connection are serviced by different threads than the actual parent connection.
2018-04-27 12:49:42 +08:00
lws_context_unlock(context); /* ====================================> */
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
#endif
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
client fixups after esp8266
2016-08-07 08:33:08 +08:00
/* clients acquire the ah and then insert themselves in fds table... */
windows: cleanup wrong and duplicated socket validity helpers https://github.com/warmcat/libwebsockets/issues/1259
2018-05-06 07:19:21 +08:00
if (wsi->position_in_fds_table != LWS_NO_FDS_POS) {
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_info("%s: Enabling %s POLLIN\n", __func__, lws_wsi_tag(wsi));
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
/* he has been stuck waiting for an ah, but now his wait is
* over, let him progress */
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
client fixups after esp8266
2016-08-07 08:33:08 +08:00
_lws_change_pollfd(wsi, 0, LWS_POLLIN, &pa);
}
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
/* point prev guy to next guy in list instead */
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
*pwsi_eligible = wsi->http.ah_wait_list;
improve timeout and ah list comments Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-25 21:39:01 +08:00
/* the guy who got one is out of the list */
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah_wait_list = NULL;
pt->http.ah_wait_list_length--;
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
clean: internally use LWS_WITH_CLIENT and _SERVER Remove some more things in LWS_WITH_SERVER=0 case
2019-08-18 05:04:15 +01:00
#if defined(LWS_WITH_CLIENT)
refactor mode and states into roles and states
2018-04-02 11:55:17 +08:00
if (lwsi_role_client(wsi) && lwsi_state(wsi) == LRS_UNCONNECTED) {
HTTP_PROXY: make usable
2017-06-12 13:36:24 +08:00
lws_pt_unlock(pt);
refactor: split out adoption and client apis to core - split raw role into separate skt and file - remove all special knowledge from the adoption apis and migrate to core - remove all special knowledge from client_connect stuff, and have it discovered by iterating the role callbacks to let those choose how to bind; migrate to core - retire the old deprecated client apis pre- client_connect_info
2018-05-10 16:13:26 +08:00
if (!lws_http_client_connect_via_info2(wsi)) {
client handle connection fail at second phase properly Signed-off-by: Andy Green <andy@warmcat.com>
2016-05-08 16:58:18 +08:00
/* our client connect has failed, the wsi
* has been closed
*/
return -1;
}
HTTP_PROXY: make usable
2017-06-12 13:36:24 +08:00
return 0;
}
client accept connection request even if no free ah It can join the free ah list and pick up client connect processing later when the ah becomes available; this simplifies the code doing the request since he won't have to deal with unexpected failures / retries based on dynamic ah availability. To do this though we have to handle that the connect_info members may not have scope that lets them still exist after we return from the first connect call, we stash them in a malloc'd buffer so the connect processing can have them much later even so. Signed-off-by: Andy Green <andy.green@linaro.org>
2016-02-29 14:19:16 +08:00
#endif
clean: general whitespace cleanup
2018-11-23 08:47:56 +08:00
assert(!!pt->http.ah_wait_list_length ==
!!(lws_intptr_t)pt->http.ah_wait_list);
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
bail:
lws_lifecycle This adds some new objects and helpers for keeping and logging info on grouped allocations, a group is, eg, SS handles or client wsis. Allocated objects get a context-unique "tag" string intended to replace %p / wsi pointers etc. Pointers quickly become confusing when allocations are freed and reused, the tag string won't repeat until you produce 2^64 objects in a context. In addition the tag string documents the object group, with prefixes like "wsi-" or "vh-" and contain object-specific additional information like the vhost name, address / port or the role of the wsi. At creation time the lws code can use a format string and args to add whatever group-specific info makes sense, eg, a wsi bound to a secure stream can also append the guid of the secure stream, it's copied into the new object tag and so is still available cleanly after the stream is destroyed if the wsi outlives it.
2020-12-25 05:54:19 +00:00
lwsl_info("%s: %s: ah %p (tsi=%d, count = %d)\n", __func__,
lws_wsi_tag(wsi), (void *)ah, pt->tid, pt->http.ah_count_in_use);
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
Subject: [PATCH] Use custom allocator Signed-off-by: Alejandro Mery <amery@geeks.cl>
2014-12-04 23:59:35 +01:00
return 0;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
nobody_usable_waiting:
lwsl_info("%s: nobody usable waiting\n", __func__);
ah pool: change to dynamic linked list For some targets like ESP32, the ah pool is mainly sitting idle wasting memory. For HTTP/2, if the client sends a series of pipelined headers on different SIDs that exist simultaneously, there is no way to stall the headers to wait for an ah, because we must read the stream for stuff like WINDOW_UPDATE on the other streams. In both these cases having the ability to free unused ah completely and allocate more dynamically if there is memory is useful, so this patch makes the ah pool an initially-empty linked list that allocates on demand up to the "max pool size" limit from the context info. When nobody wants an ah, it is freed (if someone was waiting for it, it is directly reused). For ESP32 it means no large, permanent alloc when lws starts and dynamic alloc according to how many streams the client opens, which can be controlled by SETTINGS.
2017-10-06 16:07:57 +08:00
_lws_destroy_ah(pt, ah);
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
pt->http.ah_count_in_use--;
LWS_WITH_PEER_LIMITS Introduces an optional, settable restriction on the number of simultaneous wsi connections and ah that a single peer IP can obtain. The default is disabled for build, and if enabled, unlimited. However when enabled at CMake, setting info.ip_limit_ah and / or info.ip_limit_wsi at context creation time will enforce the limits. Connections past the info.ip_limit_wsi limit are dropped, and connections needing an ah are forced to stay in the ah waiting list even when ah are available, while the peer is at the number of ah in info.ip_limit_ah.
2017-09-14 13:14:11 +08:00
goto bail;
http header malloc pool implement pool Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-25 12:44:12 +08:00
}
Adding header table free function to make sure we free cleanly everywhere.
2014-11-07 11:20:59 +08:00
pthreads: more locking
2018-03-05 16:49:28 +08:00
int lws_header_table_detach(struct lws *wsi, int autoservice)
{
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
struct lws_context *context = wsi->a.context;
pthreads: more locking
2018-03-05 16:49:28 +08:00
struct lws_context_per_thread *pt = &context->pt[(int)wsi->tsi];
int n;
lws_pt_lock(pt, __func__);
n = __lws_header_table_detach(wsi, autoservice);
lws_pt_unlock(pt);
return n;
}
cleaning
2020-01-02 08:32:23 +00:00
int
uridecoding lws_hdr_fragment_length Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-19 07:35:23 +08:00
lws_hdr_fragment_length(struct lws *wsi, enum lws_token_indexes h, int frag_idx)
{
int n;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
if (!wsi->http.ah)
lws-meta
2017-07-19 04:39:14 +08:00
return 0;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
n = wsi->http.ah->frag_index[h];
uridecoding lws_hdr_fragment_length Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-19 07:35:23 +08:00
if (!n)
return 0;
do {
if (!frag_idx)
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
return wsi->http.ah->frags[n].len;
n = wsi->http.ah->frags[n].nfrag;
uridecoding lws_hdr_fragment_length Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-19 07:35:23 +08:00
} while (frag_idx-- && n);
return 0;
}
cleaning
2020-01-02 08:32:23 +00:00
int lws_hdr_total_length(struct lws *wsi, enum lws_token_indexes h)
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
{
int n;
int len = 0;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
if (!wsi->http.ah)
lws-meta
2017-07-19 04:39:14 +08:00
return 0;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
n = wsi->http.ah->frag_index[h];
refactor out most NO_EXTENSIONS Signed-off-by: Andy Green <andy.green@linaro.org>
2014-04-02 19:45:42 +08:00
if (!n)
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
return 0;
do {
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
len += wsi->http.ah->frags[n].len;
n = wsi->http.ah->frags[n].nfrag;
lws_hdr_total_length: match lws_hdr_copy actual length also for COOKIE
2018-11-15 15:35:22 +08:00
http: hdr: sync lws_hdr_total_length with lws_hdr_copy
2021-08-27 05:47:31 +01:00
if (n)
len++;
lws_hdr_total_length: match lws_hdr_copy actual length also for COOKIE
2018-11-15 15:35:22 +08:00
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
} while (n);
return len;
}
cleaning
2020-01-02 08:32:23 +00:00
int lws_hdr_copy_fragment(struct lws *wsi, char *dst, int len,
introduce lws_hdr_copy_fragment This adds a public API variant of the header copy api that lets you choose which fragment you want copied. Normally you want the existing one that aggregates the fragments. But it can be useful to get each part in turn (that corresponds to the content provided by each duplicated header normally). Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 22:59:23 +08:00
enum lws_token_indexes h, int frag_idx)
{
int n = 0;
lws-meta
2017-07-19 04:39:14 +08:00
int f;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
if (!wsi->http.ah)
lws-meta
2017-07-19 04:39:14 +08:00
return -1;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
f = wsi->http.ah->frag_index[h];
introduce lws_hdr_copy_fragment This adds a public API variant of the header copy api that lets you choose which fragment you want copied. Normally you want the existing one that aggregates the fragments. But it can be useful to get each part in turn (that corresponds to the content provided by each duplicated header normally). Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 22:59:23 +08:00
header fragment reject empty early Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-20 09:44:04 +08:00
if (!f)
return -1;
introduce lws_hdr_copy_fragment This adds a public API variant of the header copy api that lets you choose which fragment you want copied. Normally you want the existing one that aggregates the fragments. But it can be useful to get each part in turn (that corresponds to the content provided by each duplicated header normally). Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 22:59:23 +08:00
while (n < frag_idx) {
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
f = wsi->http.ah->frags[f].nfrag;
introduce lws_hdr_copy_fragment This adds a public API variant of the header copy api that lets you choose which fragment you want copied. Normally you want the existing one that aggregates the fragments. But it can be useful to get each part in turn (that corresponds to the content provided by each duplicated header normally). Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 22:59:23 +08:00
if (!f)
return -1;
n++;
}
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
if (wsi->http.ah->frags[f].len >= len)
lws_get_urlarg_by_name: drop candidates that wont fit in buf rather than bail https://github.com/warmcat/libwebsockets/issues/3227
2024-09-28 06:01:36 +01:00
return -2;
introduce lws_hdr_copy_fragment This adds a public API variant of the header copy api that lets you choose which fragment you want copied. Normally you want the existing one that aggregates the fragments. But it can be useful to get each part in turn (that corresponds to the content provided by each duplicated header normally). Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 22:59:23 +08:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
memcpy(dst, wsi->http.ah->data + wsi->http.ah->frags[f].offset,
wsi->http.ah->frags[f].len);
dst[wsi->http.ah->frags[f].len] = '\0';
introduce lws_hdr_copy_fragment This adds a public API variant of the header copy api that lets you choose which fragment you want copied. Normally you want the existing one that aggregates the fragments. But it can be useful to get each part in turn (that corresponds to the content provided by each duplicated header normally). Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 22:59:23 +08:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
return wsi->http.ah->frags[f].len;
introduce lws_hdr_copy_fragment This adds a public API variant of the header copy api that lets you choose which fragment you want copied. Normally you want the existing one that aggregates the fragments. But it can be useful to get each part in turn (that corresponds to the content provided by each duplicated header normally). Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 22:59:23 +08:00
}
cleaning
2020-01-02 08:32:23 +00:00
int lws_hdr_copy(struct lws *wsi, char *dst, int len,
clean tidy the worst whitespace alignment probs due to mass token name length changes Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-04 11:30:53 +08:00
enum lws_token_indexes h)
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
{
http: hdr: sync lws_hdr_total_length with lws_hdr_copy
2021-08-27 05:47:31 +01:00
int toklen = lws_hdr_total_length(wsi, h), n, comma;
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
lws_tokenize
2018-10-09 10:29:42 +08:00
*dst = '\0';
if (!toklen)
return 0;
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
if (toklen >= len)
return -1;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
if (!wsi->http.ah)
lws-meta
2017-07-19 04:39:14 +08:00
return -1;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
n = wsi->http.ah->frag_index[h];
refactor out most NO_EXTENSIONS Signed-off-by: Andy Green <andy.green@linaro.org>
2014-04-02 19:45:42 +08:00
if (!n)
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
return 0;
do {
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
comma = (wsi->http.ah->frags[n].nfrag) ? 1 : 0;
http: proxy: fix uri relisting
2021-01-27 15:14:03 +00:00
if (h == WSI_TOKEN_HTTP_URI_ARGS)
http: hdr: sync lws_hdr_total_length with lws_hdr_copy
2021-08-27 05:47:31 +01:00
lwsl_notice("%s: WSI_TOKEN_HTTP_URI_ARGS '%.*s'\n",
__func__, (int)wsi->http.ah->frags[n].len,
&wsi->http.ah->data[
wsi->http.ah->frags[n].offset]);
lws_hdr_total_length: match lws_hdr_copy actual length also for COOKIE
2018-11-15 15:35:22 +08:00
http: proxy: fix uri relisting
2021-01-27 15:14:03 +00:00
if (wsi->http.ah->frags[n].len + comma >= len) {
lwsl_notice("blowout len\n");
lws_hdr_copy: protect against garbage
2017-11-02 08:10:41 +08:00
return -1;
http: proxy: fix uri relisting
2021-01-27 15:14:03 +00:00
}
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
strncpy(dst, &wsi->http.ah->data[wsi->http.ah->frags[n].offset],
wsi->http.ah->frags[n].len);
dst += wsi->http.ah->frags[n].len;
len -= wsi->http.ah->frags[n].len;
n = wsi->http.ah->frags[n].nfrag;
fulltext search
2018-10-14 11:38:08 +08:00
http: hdr: sync lws_hdr_total_length with lws_hdr_copy
2021-08-27 05:47:31 +01:00
/*
* Note if you change this logic, take care about updating len
* and make sure lws_hdr_total_length() gives the same resulting
* length
*/
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
if (comma) {
http: hdr: sync lws_hdr_total_length with lws_hdr_copy
2021-08-27 05:47:31 +01:00
if (h == WSI_TOKEN_HTTP_COOKIE ||
h == WSI_TOKEN_HTTP_SET_COOKIE)
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
*dst++ = ';';
else
http: proxy: fix uri relisting
2021-01-27 15:14:03 +00:00
if (h == WSI_TOKEN_HTTP_URI_ARGS)
*dst++ = '&';
else
*dst++ = ',';
http: hdr: sync lws_hdr_total_length with lws_hdr_copy
2021-08-27 05:47:31 +01:00
len--;
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
}
lws_hdr_total_length: match lws_hdr_copy actual length also for COOKIE
2018-11-15 15:35:22 +08:00
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
} while (n);
lws_hdr_copy: protect against garbage
2017-11-02 08:10:41 +08:00
*dst = '\0';
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
http: proxy: fix uri relisting
2021-01-27 15:14:03 +00:00
if (h == WSI_TOKEN_HTTP_URI_ARGS)
lwsl_err("%s: WSI_TOKEN_HTTP_URI_ARGS toklen %d\n", __func__, (int)toklen);
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
return toklen;
}
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
cleaning
2020-01-02 08:32:23 +00:00
int
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
lws_hdr_custom_length(struct lws *wsi, const char *name, int nlen)
{
ah_data_idx_t ll;
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (!wsi->http.ah || wsi->mux_substream)
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
return -1;
ll = wsi->http.ah->unk_ll_head;
while (ll) {
if (ll >= wsi->http.ah->data_length)
return -1;
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
if (nlen == lws_ser_ru16be(
(uint8_t *)&wsi->http.ah->data[ll + UHO_NLEN]) &&
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
!strncmp(name, &wsi->http.ah->data[ll + UHO_NAME], (unsigned int)nlen))
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
return lws_ser_ru16be(
(uint8_t *)&wsi->http.ah->data[ll + UHO_VLEN]);
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
ll = lws_ser_ru32be((uint8_t *)&wsi->http.ah->data[ll + UHO_LL]);
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
}
return -1;
}
cleaning
2020-01-02 08:32:23 +00:00
int
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
lws_hdr_custom_copy(struct lws *wsi, char *dst, int len, const char *name,
int nlen)
{
ah_data_idx_t ll;
int n;
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (!wsi->http.ah || wsi->mux_substream)
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
return -1;
*dst = '\0';
ll = wsi->http.ah->unk_ll_head;
while (ll) {
if (ll >= wsi->http.ah->data_length)
return -1;
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
if (nlen == lws_ser_ru16be(
(uint8_t *)&wsi->http.ah->data[ll + UHO_NLEN]) &&
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
!strncmp(name, &wsi->http.ah->data[ll + UHO_NAME], (unsigned int)nlen)) {
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
n = lws_ser_ru16be(
(uint8_t *)&wsi->http.ah->data[ll + UHO_VLEN]);
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
if (n + 1 > len)
return -1;
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
strncpy(dst, &wsi->http.ah->data[ll + UHO_NAME + (unsigned int)nlen], (unsigned int)n);
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
dst[n] = '\0';
return n;
}
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
ll = lws_ser_ru32be((uint8_t *)&wsi->http.ah->data[ll + UHO_LL]);
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
}
return -1;
}
http: custom hdr: add custom name iterator
2021-08-28 06:18:32 +01:00
int
lws_hdr_custom_name_foreach(struct lws *wsi, lws_hdr_custom_fe_cb_t cb,
void *custom)
{
ah_data_idx_t ll;
if (!wsi->http.ah || wsi->mux_substream)
return -1;
ll = wsi->http.ah->unk_ll_head;
while (ll) {
if (ll >= wsi->http.ah->data_length)
return -1;
cb(&wsi->http.ah->data[ll + UHO_NAME],
lws_ser_ru16be((uint8_t *)&wsi->http.ah->data[ll + UHO_NLEN]),
custom);
ll = lws_ser_ru32be((uint8_t *)&wsi->http.ah->data[ll + UHO_LL]);
}
return 0;
}
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#endif
api rationalization: eliminate all libwebsocket[s]_ prefixes This nukes all the oldstyle prefixes except in the compatibility code. struct libwebsockets becomes struct lws too. The api docs are updated accordingly as are the READMEs that mention those apis. Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-04 11:08:32 +08:00
char *lws_hdr_simple_ptr(struct lws *wsi, enum lws_token_indexes h)
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
{
int n;
http: compression methods Add generic http compression layer eanbled at cmake with LWS_WITH_HTTP_STREAM_COMPRESSION. This is wholly a feature of the HTTP role (used by h1 and h2 roles) and doesn't exist outside that context. Currently provides 'deflate' and 'br' compression methods for server side only. 'br' requires also -DLWS_WITH_HTTP_BROTLI=1 at cmake and the brotli libraries (available in your distro already) and dev package. Other compression methods can be added nicely using an ops struct. The built-in file serving stuff will use this is the client says he can handle it, and the mimetype of the file either starts with "text/" (html and css etc) or is the mimetype of Javascript. zlib allocates quite a bit while in use, it seems to be around 256KiB per stream. So this is only useful on relatively strong servers with lots of memory. However for some usecases where you are serving a lot of css and js assets, it's a nice help. The patch performs special treatment for http/1.1 pipelining, since the compression is performed on the fly the compressed content-length is not known until the end. So for h1 only, chunked transfer-encoding is automatically added so pipelining can continue of the connection. For h2 the chunking is neither supported nor required, so it "just works". User code can also request to add a compression transform before the reply headers were sent using the new api LWS_VISIBLE int lws_http_compression_apply(struct lws *wsi, const char *name, unsigned char **p, unsigned char *end, char decomp); ... this allows transparent compression of dynamically generated HTTP. The requested compression (eg, "deflate") is only applied if the client headers indicated it was supported, otherwise it's a NOP. Name may be NULL in which case the first compression method in the internal table at stream.c that is mentioned as acceptable by the client will be used. NOTE: the compression translation, same as h2 support, relies on the user code using LWS_WRITE_HTTP and then LWS_WRITE_HTTP_FINAL on the last part written. The internal lws fileserving code already does this.
2018-09-02 14:43:05 +08:00
if (!wsi->http.ah)
return NULL;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
n = wsi->http.ah->frag_index[h];
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
if (!n)
return NULL;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
return wsi->http.ah->data + wsi->http.ah->frags[n].offset;
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
}
add lexical parser for headers Profiling what happens during the ab test, one of the hotspots was strcasecmp in a loop looking for header name matches each time. This patch introduces a lexical parser that creates a state machine in 276 bytes that encodes all the known header names. The fsm is walked bytewise as chaacters come in... most states do not need any recursion to match or fail. The state machine output is cut-and-pasted into parsers.c as an unsigned char array. The fsm generator is a bit rough and ready, included in the tree but not built since normal mortals won't need to touch it. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-01-18 01:55:48 +08:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
static int LWS_WARN_UNUSED_RESULT
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
lws_pos_in_bounds(struct lws *wsi)
{
http: compression methods Add generic http compression layer eanbled at cmake with LWS_WITH_HTTP_STREAM_COMPRESSION. This is wholly a feature of the HTTP role (used by h1 and h2 roles) and doesn't exist outside that context. Currently provides 'deflate' and 'br' compression methods for server side only. 'br' requires also -DLWS_WITH_HTTP_BROTLI=1 at cmake and the brotli libraries (available in your distro already) and dev package. Other compression methods can be added nicely using an ops struct. The built-in file serving stuff will use this is the client says he can handle it, and the mimetype of the file either starts with "text/" (html and css etc) or is the mimetype of Javascript. zlib allocates quite a bit while in use, it seems to be around 256KiB per stream. So this is only useful on relatively strong servers with lots of memory. However for some usecases where you are serving a lot of css and js assets, it's a nice help. The patch performs special treatment for http/1.1 pipelining, since the compression is performed on the fly the compressed content-length is not known until the end. So for h1 only, chunked transfer-encoding is automatically added so pipelining can continue of the connection. For h2 the chunking is neither supported nor required, so it "just works". User code can also request to add a compression transform before the reply headers were sent using the new api LWS_VISIBLE int lws_http_compression_apply(struct lws *wsi, const char *name, unsigned char **p, unsigned char *end, char decomp); ... this allows transparent compression of dynamically generated HTTP. The requested compression (eg, "deflate") is only applied if the client headers indicated it was supported, otherwise it's a NOP. Name may be NULL in which case the first compression method in the internal table at stream.c that is mentioned as acceptable by the client will be used. NOTE: the compression translation, same as h2 support, relies on the user code using LWS_WRITE_HTTP and then LWS_WRITE_HTTP_FINAL on the last part written. The internal lws fileserving code already does this.
2018-09-02 14:43:05 +08:00
if (!wsi->http.ah)
return -1;
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
if (wsi->http.ah->pos <
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
(unsigned int)wsi->a.context->max_http_header_data)
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
return 0;
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
if ((int)wsi->http.ah->pos >= (int)wsi->a.context->max_http_header_data - 1) {
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
lwsl_err("Ran out of header data space\n");
return 1;
}
/*
* with these tests everywhere, it should never be able to exceed
clean
2017-09-23 12:55:21 +08:00
* the limit, only meet it
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
*/
rtos: clean warnings where uint32_t is an unsigned long
2019-06-07 11:11:46 +01:00
lwsl_err("%s: pos %ld, limit %ld\n", __func__,
(unsigned long)wsi->http.ah->pos,
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
(unsigned long)wsi->a.context->max_http_header_data);
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
assert(0);
return 1;
}
int LWS_WARN_UNUSED_RESULT
lws_hdr_simple_create(struct lws *wsi, enum lws_token_indexes h, const char *s)
client convert to new headers scheme eliminating mallocs This removes all the direct wsi members specific to clients, most of them are moved to being fake headers in the next 3-layer header scheme, c_port moves to being a member of the u.hdr unionized struct. It gets rid of a lot of fiddly mallocs and frees(), despite it adds a small internal API to create the fake headers, actually the patch deletes more than it adds... Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 13:04:45 +08:00
{
ah: simple_create an empty string removes header entry Length is zero either way https://github.com/warmcat/libwebsockets/issues/1820
2020-01-10 10:05:35 +00:00
if (!*s) {
/*
* If we get an empty string, then remove any entry for the
* header
*/
wsi->http.ah->frag_index[h] = 0;
return 0;
}
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah->nfrag++;
libwebsockets.h: clean out some boilerplate better put in core/private.h https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
if (wsi->http.ah->nfrag == LWS_ARRAY_SIZE(wsi->http.ah->frags)) {
style cleanup This brings the library sources into compliance with checkpatch style except for three or four exceptions like WIN32 related stuff and one long string constant I don't want to break into multiple sprintf calls. There should be no functional or compilability change from all this (hopefully). Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 17:13:32 +08:00
lwsl_warn("More hdr frags than we can deal with, dropping\n");
client convert to new headers scheme eliminating mallocs This removes all the direct wsi members specific to clients, most of them are moved to being fake headers in the next 3-layer header scheme, c_port moves to being a member of the u.hdr unionized struct. It gets rid of a lot of fiddly mallocs and frees(), despite it adds a small internal API to create the fake headers, actually the patch deletes more than it adds... Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 13:04:45 +08:00
return -1;
}
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah->frag_index[h] = wsi->http.ah->nfrag;
client convert to new headers scheme eliminating mallocs This removes all the direct wsi members specific to clients, most of them are moved to being fake headers in the next 3-layer header scheme, c_port moves to being a member of the u.hdr unionized struct. It gets rid of a lot of fiddly mallocs and frees(), despite it adds a small internal API to create the fake headers, actually the patch deletes more than it adds... Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 13:04:45 +08:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah->frags[wsi->http.ah->nfrag].offset = wsi->http.ah->pos;
wsi->http.ah->frags[wsi->http.ah->nfrag].len = 0;
wsi->http.ah->frags[wsi->http.ah->nfrag].nfrag = 0;
client convert to new headers scheme eliminating mallocs This removes all the direct wsi members specific to clients, most of them are moved to being fake headers in the next 3-layer header scheme, c_port moves to being a member of the u.hdr unionized struct. It gets rid of a lot of fiddly mallocs and frees(), despite it adds a small internal API to create the fake headers, actually the patch deletes more than it adds... Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 13:04:45 +08:00
do {
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
if (lws_pos_in_bounds(wsi))
client convert to new headers scheme eliminating mallocs This removes all the direct wsi members specific to clients, most of them are moved to being fake headers in the next 3-layer header scheme, c_port moves to being a member of the u.hdr unionized struct. It gets rid of a lot of fiddly mallocs and frees(), despite it adds a small internal API to create the fake headers, actually the patch deletes more than it adds... Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 13:04:45 +08:00
return -1;
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah->data[wsi->http.ah->pos++] = *s;
client convert to new headers scheme eliminating mallocs This removes all the direct wsi members specific to clients, most of them are moved to being fake headers in the next 3-layer header scheme, c_port moves to being a member of the u.hdr unionized struct. It gets rid of a lot of fiddly mallocs and frees(), despite it adds a small internal API to create the fake headers, actually the patch deletes more than it adds... Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 13:04:45 +08:00
if (*s)
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah->frags[wsi->http.ah->nfrag].len++;
client convert to new headers scheme eliminating mallocs This removes all the direct wsi members specific to clients, most of them are moved to being fake headers in the next 3-layer header scheme, c_port moves to being a member of the u.hdr unionized struct. It gets rid of a lot of fiddly mallocs and frees(), despite it adds a small internal API to create the fake headers, actually the patch deletes more than it adds... Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 13:04:45 +08:00
} while (*s++);
return 0;
}
introduce LWS_WARN_UNUSED_RESULT Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-20 16:56:06 +08:00
static int LWS_WARN_UNUSED_RESULT
issue_char(struct lws *wsi, unsigned char c)
translate and protect uri test sever use uri path This translates %xx in the GET uri and removes /.. and /... type sequences along with translating // or /// etc to /. Since the result is hopefully secure, it also changes the test server to actually use the uri path pasted on a resource directory without whitelisting. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-11-10 15:15:21 +08:00
{
C89 tweaks as per #348.
2015-11-29 19:26:01 +08:00
unsigned short frag_len;
refactor needless context with wsi paramater passing Now we bit the bullet and gave each wsi an lws_context *, many internal apis that take both a context and wsi parameter only need the wsi. Also simplify parser code by making a temp var for allocated_headers * instead of the longwinded dereference chain everywhere. Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 21:15:58 +08:00
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
if (lws_pos_in_bounds(wsi))
translate and protect uri test sever use uri path This translates %xx in the GET uri and removes /.. and /... type sequences along with translating // or /// etc to /. Since the result is hopefully secure, it also changes the test server to actually use the uri path pasted on a resource directory without whitelisting. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-11-10 15:15:21 +08:00
return -1;
Added optional per-header length limits: - libwebsockets.h: - * added struct lws_token_limits * added token limits pointer to lws_context_creation_info - private-libwebsockets.h: added token limits pointer to lws_context_creation_info - context.c: copy token limits in create_context - client.c / server.c: pass context when invoking libwebsocket_parse - parsers.c: - * libwebsocket_parse takes context pointer * issue_char takes context pointer and checks header length against context limits, if defined * issue_char returns 1 (not -1/0) for header too long, and spill: sets the state to WSI_TOKEN_SKIPPING
2014-06-29 00:25:19 -04:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
frag_len = wsi->http.ah->frags[wsi->http.ah->nfrag].len;
refactor needless context with wsi paramater passing Now we bit the bullet and gave each wsi an lws_context *, many internal apis that take both a context and wsi parameter only need the wsi. Also simplify parser code by making a temp var for allocated_headers * instead of the longwinded dereference chain everywhere. Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 21:15:58 +08:00
/*
* If we haven't hit the token limit, just copy the character into
* the header
*/
plugins: gitws libjsongit2 support This adds a plugin that interfaces to libjsongit2 https://warmcat.com/git/libjsongit2 to provide a per-vhost service for presenting bare git repos in a web interface.
2018-08-14 08:00:25 +08:00
if (!wsi->http.ah->current_token_limit ||
frag_len < wsi->http.ah->current_token_limit) {
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
wsi->http.ah->data[wsi->http.ah->pos++] = (char)c;
http: parser: straighten out %00 legality https://github.com/warmcat/libwebsockets/issues/2262 This adds a README explaining what can be expected if your URLs contain %00, and adds a safe helper for urlargs-by-name that is length-based. Contains fix for extra NUL on some headers https://github.com/warmcat/libwebsockets/issues/2267
2021-04-08 15:26:23 +01:00
wsi->http.ah->frags[wsi->http.ah->nfrag].len++;
Terminate truncated header strings.
2015-11-29 19:24:04 +08:00
return 0;
}
refactor needless context with wsi paramater passing Now we bit the bullet and gave each wsi an lws_context *, many internal apis that take both a context and wsi parameter only need the wsi. Also simplify parser code by making a temp var for allocated_headers * instead of the longwinded dereference chain everywhere. Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 21:15:58 +08:00
/* Insert a null character when we *hit* the limit: */
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
if (frag_len == wsi->http.ah->current_token_limit) {
unify bounds checking in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-21 10:57:39 +08:00
if (lws_pos_in_bounds(wsi))
parser issue_char audit Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-20 07:40:13 +08:00
return -1;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
wsi->http.ah->data[wsi->http.ah->pos++] = '\0';
rtos: clean warnings where uint32_t is an unsigned long
2019-06-07 11:11:46 +01:00
lwsl_warn("header %li exceeds limit %ld\n",
(long)wsi->http.ah->parser_state,
(long)wsi->http.ah->current_token_limit);
refactor needless context with wsi paramater passing Now we bit the bullet and gave each wsi an lws_context *, many internal apis that take both a context and wsi parameter only need the wsi. Also simplify parser code by making a temp var for allocated_headers * instead of the longwinded dereference chain everywhere. Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 21:15:58 +08:00
}
Terminate truncated header strings.
2015-11-29 19:24:04 +08:00
return 1;
translate and protect uri test sever use uri path This translates %xx in the GET uri and removes /.. and /... type sequences along with translating // or /// etc to /. Since the result is hopefully secure, it also changes the test server to actually use the uri path pasted on a resource directory without whitelisting. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-11-10 15:15:21 +08:00
}
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
int
lws_parse_urldecode(struct lws *wsi, uint8_t *_c)
{
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
struct allocated_headers *ah = wsi->http.ah;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
unsigned int enc = 0;
uint8_t c = *_c;
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
// lwsl_notice("ah->ups %d\n", ah->ups);
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
/*
* PRIORITY 1
* special URI processing... convert %xx
*/
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
switch (ah->ues) {
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
case URIES_IDLE:
if (c == '%') {
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ues = URIES_SEEN_PERCENT;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto swallow;
}
break;
case URIES_SEEN_PERCENT:
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
if (char_to_hex((char)c) < 0)
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
/* illegal post-% char */
goto forbid;
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->esc_stash = (char)c;
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ues = URIES_SEEN_PERCENT_H1;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto swallow;
case URIES_SEEN_PERCENT_H1:
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
if (char_to_hex((char)c) < 0)
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
/* illegal post-% char */
goto forbid;
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
*_c = (uint8_t)(unsigned int)((char_to_hex(ah->esc_stash) << 4) |
char_to_hex((char)c));
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
c = *_c;
enc = 1;
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ues = URIES_IDLE;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
break;
}
/*
* PRIORITY 2
* special URI processing...
* convert /.. or /... or /../ etc to /
* convert /./ to /
* convert // or /// etc to /
* leave /.dir or whatever alone
*/
http: parser: straighten out %00 legality https://github.com/warmcat/libwebsockets/issues/2262 This adds a README explaining what can be expected if your URLs contain %00, and adds a safe helper for urlargs-by-name that is length-based. Contains fix for extra NUL on some headers https://github.com/warmcat/libwebsockets/issues/2267
2021-04-08 15:26:23 +01:00
if (!c && (!ah->frag_index[WSI_TOKEN_HTTP_URI_ARGS] ||
!ah->post_literal_equal)) {
/*
* Since user code is typically going to parse the path using
* NUL-terminated apis, it's too dangerous to allow NUL
* injection here.
*
* It's allowed in the urlargs, because the apis to access
* those only allow retreival with explicit length.
*/
lwsl_warn("%s: saw NUL outside of uri args\n", __func__);
return -1;
}
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
switch (ah->ups) {
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
case URIPS_IDLE:
http: parser: straighten out %00 legality https://github.com/warmcat/libwebsockets/issues/2262 This adds a README explaining what can be expected if your URLs contain %00, and adds a safe helper for urlargs-by-name that is length-based. Contains fix for extra NUL on some headers https://github.com/warmcat/libwebsockets/issues/2267
2021-04-08 15:26:23 +01:00
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
/* genuine delimiter */
if ((c == '&' || c == ';') && !enc) {
http: urlarg capture fixes
2018-06-19 12:30:10 +08:00
if (issue_char(wsi, '\0') < 0)
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
return -1;
http: parser: straighten out %00 legality https://github.com/warmcat/libwebsockets/issues/2262 This adds a README explaining what can be expected if your URLs contain %00, and adds a safe helper for urlargs-by-name that is length-based. Contains fix for extra NUL on some headers https://github.com/warmcat/libwebsockets/issues/2267
2021-04-08 15:26:23 +01:00
/* don't account for it */
wsi->http.ah->frags[wsi->http.ah->nfrag].len--;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
/* link to next fragment */
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->frags[ah->nfrag].nfrag = (uint8_t)(ah->nfrag + 1);
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
ah->nfrag++;
libwebsockets.h: clean out some boilerplate better put in core/private.h https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
if (ah->nfrag >= LWS_ARRAY_SIZE(ah->frags))
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto excessive;
/* start next fragment after the & */
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->post_literal_equal = 0;
http: urlarg capture fixes
2018-06-19 12:30:10 +08:00
ah->frags[ah->nfrag].offset = ++ah->pos;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
ah->frags[ah->nfrag].len = 0;
ah->frags[ah->nfrag].nfrag = 0;
goto swallow;
}
/* uriencoded = in the name part, disallow */
if (c == '=' && enc &&
ah->frag_index[WSI_TOKEN_HTTP_URI_ARGS] &&
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
!ah->post_literal_equal) {
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
c = '_';
*_c =c;
}
/* after the real =, we don't care how many = */
if (c == '=' && !enc)
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->post_literal_equal = 1;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
/* + to space */
if (c == '+' && !enc) {
c = ' ';
*_c = c;
}
/* issue the first / always */
if (c == '/' && !ah->frag_index[WSI_TOKEN_HTTP_URI_ARGS])
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_SEEN_SLASH;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
break;
case URIPS_SEEN_SLASH:
/* swallow subsequent slashes */
if (c == '/')
goto swallow;
/* track and swallow the first . after / */
if (c == '.') {
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_SEEN_SLASH_DOT;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto swallow;
}
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_IDLE;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
break;
case URIPS_SEEN_SLASH_DOT:
/* swallow second . */
if (c == '.') {
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_SEEN_SLASH_DOT_DOT;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto swallow;
}
/* change /./ to / */
if (c == '/') {
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_SEEN_SLASH;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto swallow;
}
/* it was like /.dir ... regurgitate the . */
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_IDLE;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
if (issue_char(wsi, '.') < 0)
return -1;
break;
case URIPS_SEEN_SLASH_DOT_DOT:
/* /../ or /..[End of URI] --> backup to last / */
if (c == '/' || c == '?') {
/*
* back up one dir level if possible
* safe against header fragmentation because
* the method URI can only be in 1 fragment
*/
if (ah->frags[ah->nfrag].len > 2) {
ah->pos--;
ah->frags[ah->nfrag].len--;
do {
ah->pos--;
ah->frags[ah->nfrag].len--;
} while (ah->frags[ah->nfrag].len > 1 &&
ah->data[ah->pos] != '/');
}
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_SEEN_SLASH;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
if (ah->frags[ah->nfrag].len > 1)
break;
goto swallow;
}
/* /..[^/] ... regurgitate and allow */
if (issue_char(wsi, '.') < 0)
return -1;
if (issue_char(wsi, '.') < 0)
return -1;
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_IDLE;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
break;
}
if (c == '?' && !enc &&
refactor-eliminate-lws-union The union used to make a lot of sense to save space between mutually exclusive modes. But the fact the http2 struct contains the http1 struct as well as it appearing in the union means the http1 struct belongs outside the union. This patch - eliminates the union - puts the http_related struct directly in struct lws - removes http_related from h2 - puts h2 directly in struct lws if enabled for build - changes ws to be a pointer, allocated if we upgrade to ws (the ws part contains a 135 byte char array for ping / close) Again all of this is entirely private / internal and doesn't affect any apis.
2017-12-01 11:09:32 +08:00
!ah->frag_index[WSI_TOKEN_HTTP_URI_ARGS]) { /* start of URI args */
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
if (ah->ues != URIES_IDLE)
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto forbid;
/* seal off uri header */
if (issue_char(wsi, '\0') < 0)
return -1;
http: parser: straighten out %00 legality https://github.com/warmcat/libwebsockets/issues/2262 This adds a README explaining what can be expected if your URLs contain %00, and adds a safe helper for urlargs-by-name that is length-based. Contains fix for extra NUL on some headers https://github.com/warmcat/libwebsockets/issues/2267
2021-04-08 15:26:23 +01:00
/* don't account for it */
wsi->http.ah->frags[wsi->http.ah->nfrag].len--;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
/* move to using WSI_TOKEN_HTTP_URI_ARGS */
ah->nfrag++;
libwebsockets.h: clean out some boilerplate better put in core/private.h https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
if (ah->nfrag >= LWS_ARRAY_SIZE(ah->frags))
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto excessive;
http: skip NULs between fragments
2018-06-19 12:19:10 +08:00
ah->frags[ah->nfrag].offset = ++ah->pos;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
ah->frags[ah->nfrag].len = 0;
ah->frags[ah->nfrag].nfrag = 0;
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->post_literal_equal = 0;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
ah->frag_index[WSI_TOKEN_HTTP_URI_ARGS] = ah->nfrag;
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->ups = URIPS_IDLE;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
goto swallow;
}
return LPUR_CONTINUE;
swallow:
return LPUR_SWALLOW;
forbid:
return LPUR_FORBID;
excessive:
return LPUR_EXCESSIVE;
}
static const unsigned char methods[] = {
WSI_TOKEN_GET_URI,
WSI_TOKEN_POST_URI,
rtos diet: http: remove headers at buildtime according to config Headers related to ws or h2 are now elided if the ws or h2 role is not enabled for build. In addition, a new build-time option LWS_WITH_HTTP_UNCOMMON_HEADERS on by default allows removal of less-common http headers to shrink the parser footprint. Minilex is adapted to produce 8 different versions of the lex table, chosen at build-time according to which headers are included in the build. If you don't need the unusual headers, or aren't using h2 or ws, this chops down the size of the ah and the rodata needed to hold the parsing table from 87 strings / pointers to 49, and the parsing table from 1177 to 696 bytes.
2020-02-28 10:31:04 +00:00
#if defined(LWS_WITH_HTTP_UNCOMMON_HEADERS)
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
WSI_TOKEN_OPTIONS_URI,
WSI_TOKEN_PUT_URI,
WSI_TOKEN_PATCH_URI,
WSI_TOKEN_DELETE_URI,
rtos diet: http: remove headers at buildtime according to config Headers related to ws or h2 are now elided if the ws or h2 role is not enabled for build. In addition, a new build-time option LWS_WITH_HTTP_UNCOMMON_HEADERS on by default allows removal of less-common http headers to shrink the parser footprint. Minilex is adapted to produce 8 different versions of the lex table, chosen at build-time according to which headers are included in the build. If you don't need the unusual headers, or aren't using h2 or ws, this chops down the size of the ah and the rodata needed to hold the parsing table from 87 strings / pointers to 49, and the parsing table from 1177 to 696 bytes.
2020-02-28 10:31:04 +00:00
#endif
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
WSI_TOKEN_CONNECT,
WSI_TOKEN_HEAD_URI,
};
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/*
* possible returns:, -1 fail, 0 ok or 2, transition to raw
*/
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
lws_parser_return_t LWS_WARN_UNUSED_RESULT
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
lws_parse(struct lws *wsi, unsigned char *buf, int *len)
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
{
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
struct allocated_headers *ah = wsi->http.ah;
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
struct lws_context *context = wsi->a.context;
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
unsigned int n, m;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
unsigned char c;
int r, pos;
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
refactor: move all ah and http specific vars to http private and conditional inclusion from there
2018-04-27 15:20:56 +08:00
assert(wsi->http.ah);
multithread stability Signed-off-by: Andy Green <andy.green@linaro.org>
2016-01-26 20:56:56 +08:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
do {
(*len)--;
c = *buf++;
introduce x google mux very draft indeed This is initial x-google-mux support. It's disabled by default since it's very pre-alpha. 1) To enable it, reconfigure with --enable-x-google-mux 2) It conflicts with deflate-stream, use the -u switch on the test client to disable deflate-stream 3) It deviates from the google standard by sending full headers in the addchannel subcommand rather than just changed ones from original connect 4) Quota is not implemented yet 5) Close of subchannel is not really implemented yet 6) Google opcode 0xf is changed to 0x7 to account for v7 protocol changes to opcode layout However despite those caveats, in fact it can run the test client reliably over one socket (both dumb-increment and lws-mirror-protocol), you can open a browser on the same test server too and see the circles, etc. Signed-off-by: Andy Green <andy@warmcat.com>
2011-05-23 10:00:03 +01:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
switch (ah->parser_state) {
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
case WSI_TOKEN_UNKNOWN_VALUE_PART:
if (c == '\r')
break;
if (c == '\n') {
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
lws_ser_wu16be((uint8_t *)&ah->data[ah->unk_pos + 2],
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
(uint16_t)(ah->pos - ah->unk_value_pos));
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
ah->parser_state = WSI_TOKEN_NAME_PART;
ah->unk_pos = 0;
ah->lextable_pos = 0;
break;
}
/* trim leading whitespace */
if (ah->pos != ah->unk_value_pos ||
(c != ' ' && c != '\t')) {
if (lws_pos_in_bounds(wsi))
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
return LPR_FAIL;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->data[ah->pos++] = (char)c;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
}
pos = ah->lextable_pos;
break;
#endif
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
default:
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
lwsl_parser("WSI_TOK_(%d) '%c'\n", ah->parser_state, c);
coding-style-cleanups.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-13 10:03:47 +00:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* collect into malloc'd buffers */
/* optional initial space swallow */
if (!ah->frags[ah->frag_index[ah->parser_state]].len &&
c == ' ')
Add PATCH, PUT, DELETE methods AG: rewrite method code to use method arrays
2015-01-10 19:01:52 -08:00
break;
translate and protect uri test sever use uri path This translates %xx in the GET uri and removes /.. and /... type sequences along with translating // or /// etc to /. Since the result is hopefully secure, it also changes the test server to actually use the uri path pasted on a resource directory without whitelisting. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-11-10 15:15:21 +08:00
libwebsockets.h: clean out some boilerplate better put in core/private.h https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
for (m = 0; m < LWS_ARRAY_SIZE(methods); m++)
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (ah->parser_state == methods[m])
break;
libwebsockets.h: clean out some boilerplate better put in core/private.h https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
if (m == LWS_ARRAY_SIZE(methods))
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* it was not any of the methods */
goto check_eol;
translate and protect uri test sever use uri path This translates %xx in the GET uri and removes /.. and /... type sequences along with translating // or /// etc to /. Since the result is hopefully secure, it also changes the test server to actually use the uri path pasted on a resource directory without whitelisting. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-11-10 15:15:21 +08:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* special URI processing... end at space */
HTTP Version, Keep-alive support, No-copy POST This is a squashed commit from https://github.com/andrew-canaday/libwebsockets, dev/http_keepalive branch (strategies changed a few times, so the commit history is clutteread). This branch is submitted for clarity, but the other can be used as a reference or alternative. * added **enum http_version** to track HTTP/1.0 vs HTTP/1.1 requests * added **enum http_connection_type** to track keep-alive vs close * replaced content_length_seen and body_index with **content_remain** * removed **post_buffer** (see handshake.c modifications) * removed post_buffer free * switch state to WSI_TOKEN_SKIPPING after URI is complete to store version * delete *spill* label (unused) * add vars to track HTTP version and connection type * HTTP version defaults to 1.0 * connection type defaults to 'close' for 1.0, keep-alive for 1.1 * additional checks in **cleanup:** label: * if HTTP version string is present and valid, set enum val appropriately * override connection default with the "Connection:" header, if present * set state to WSI_STATE_HTTP_BODY if content_length > 0 * return 0 on HTTP requests, unless LWS_CALLBACK_HTTP indicates otherwise * add vars to track remaining content_length and body chunk size * re-arrange switch case order to facilitate creation of jump-table * added new labels: * **read_ok**: normal location reach on break from switch; just return 0 * **http_complete**: check for keep-alive + init state, mode, hdr table * **http_new**: jump location for keep-alive when http_complete sees len>0 * after libwebsocket_parse, jump to one of those labels based on state * POST body handling: * don't bother iterating over input byte-by-byte or using memcpy * just pass the relevant portion of the context->service_buffer to callback
2014-07-13 01:07:36 -04:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (c == ' ') {
/* enforce starting with / */
if (!ah->frags[ah->nfrag].len)
if (issue_char(wsi, '/') < 0)
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
return LPR_FAIL;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (ah->ups == URIPS_SEEN_SLASH_DOT_DOT) {
/*
* back up one dir level if possible
clean: general whitespace cleanup
2018-11-23 08:47:56 +08:00
* safe against header fragmentation
* because the method URI can only be
* in 1 fragment
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
*/
if (ah->frags[ah->nfrag].len > 2) {
urldecode forbid malformed And update attack.sh to confirm the new test cases Signed-off-by: Andy Green <andy@warmcat.com>
2016-04-07 10:08:35 +08:00
ah->pos--;
ah->frags[ah->nfrag].len--;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
do {
ah->pos--;
ah->frags[ah->nfrag].len--;
} while (ah->frags[ah->nfrag].len > 1 &&
ah->data[ah->pos] != '/');
}
urldecode forbid malformed And update attack.sh to confirm the new test cases Signed-off-by: Andy Green <andy@warmcat.com>
2016-04-07 10:08:35 +08:00
}
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* begin parsing HTTP version: */
if (issue_char(wsi, '\0') < 0)
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
return LPR_FAIL;
http: parser: straighten out %00 legality https://github.com/warmcat/libwebsockets/issues/2262 This adds a README explaining what can be expected if your URLs contain %00, and adds a safe helper for urlargs-by-name that is length-based. Contains fix for extra NUL on some headers https://github.com/warmcat/libwebsockets/issues/2267
2021-04-08 15:26:23 +01:00
/* don't account for it */
wsi->http.ah->frags[wsi->http.ah->nfrag].len--;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
ah->parser_state = WSI_TOKEN_HTTP;
goto start_fragment;
urldecode forbid malformed And update attack.sh to confirm the new test cases Signed-off-by: Andy Green <andy@warmcat.com>
2016-04-07 10:08:35 +08:00
}
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
r = lws_parse_urldecode(wsi, &c);
switch (r) {
case LPUR_CONTINUE:
break;
case LPUR_SWALLOW:
goto swallow;
case LPUR_FORBID:
goto forbid;
case LPUR_EXCESSIVE:
goto excessive;
default:
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
return LPR_FAIL;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
}
Only check for '?' URI args during URI path parse
2014-07-19 06:58:53 +08:00
check_eol:
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* bail at EOL */
if (ah->parser_state != WSI_TOKEN_CHALLENGE &&
ah: workaround for broken routers with no http header delimiter There's a type of router in the wild issuing malformed http when in captive portal mode... there's no \x0a\x0d but just \x0a
2020-06-30 17:50:01 +01:00
(c == '\x0d' || c == '\x0a')) {
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (ah->ues != URIES_IDLE)
goto forbid;
ah: workaround for broken routers with no http header delimiter There's a type of router in the wild issuing malformed http when in captive portal mode... there's no \x0a\x0d but just \x0a
2020-06-30 17:50:01 +01:00
if (c == '\x0a') {
/* broken peer */
ah->parser_state = WSI_TOKEN_NAME_PART;
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->unk_pos = 0;
ah->lextable_pos = 0;
ah: workaround for broken routers with no http header delimiter There's a type of router in the wild issuing malformed http when in captive portal mode... there's no \x0a\x0d but just \x0a
2020-06-30 17:50:01 +01:00
} else
ah->parser_state = WSI_TOKEN_SKIPPING_SAW_CR;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
c = '\0';
lwsl_parser("*\n");
}
Only check for '?' URI args during URI path parse
2014-07-19 06:58:53 +08:00
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
n = (unsigned int)issue_char(wsi, c);
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if ((int)n < 0)
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
return LPR_FAIL;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (n > 0)
ah->parser_state = WSI_TOKEN_SKIPPING;
http: parser: straighten out %00 legality https://github.com/warmcat/libwebsockets/issues/2262 This adds a README explaining what can be expected if your URLs contain %00, and adds a safe helper for urlargs-by-name that is length-based. Contains fix for extra NUL on some headers https://github.com/warmcat/libwebsockets/issues/2267
2021-04-08 15:26:23 +01:00
else {
/*
* Explicit zeroes are legal in URI ARGS.
* They can only exist as a safety terminator
* after the valid part of the token contents
* for other types.
*/
if (!c && ah->parser_state != WSI_TOKEN_HTTP_URI_ARGS)
/* don't account for safety terminator */
wsi->http.ah->frags[wsi->http.ah->nfrag].len--;
}
cleanup style in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2014-08-19 18:34:31 +08:00
translate and protect uri test sever use uri path This translates %xx in the GET uri and removes /.. and /... type sequences along with translating // or /// etc to /. Since the result is hopefully secure, it also changes the test server to actually use the uri path pasted on a resource directory without whitelisting. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-11-10 15:15:21 +08:00
swallow:
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* per-protocol end of headers management */
introduce-new-04-headers-logic.patch Signed-off-by: Andy Green <andy@warmcat.com>
2011-01-18 15:39:02 +00:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (ah->parser_state == WSI_TOKEN_CHALLENGE)
goto set_parsing_complete;
break;
introduce x google mux very draft indeed This is initial x-google-mux support. It's disabled by default since it's very pre-alpha. 1) To enable it, reconfigure with --enable-x-google-mux 2) It conflicts with deflate-stream, use the -u switch on the test client to disable deflate-stream 3) It deviates from the google standard by sending full headers in the addchannel subcommand rather than just changed ones from original connect 4) Quota is not implemented yet 5) Close of subchannel is not really implemented yet 6) Google opcode 0xf is changed to 0x7 to account for v7 protocol changes to opcode layout However despite those caveats, in fact it can run the test client reliably over one socket (both dumb-increment and lws-mirror-protocol), you can open a browser on the same test server too and see the circles, etc. Signed-off-by: Andy Green <andy@warmcat.com>
2011-05-23 10:00:03 +01:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* collecting and checking a name part */
case WSI_TOKEN_NAME_PART:
rtos: clean warnings where uint32_t is an unsigned long
2019-06-07 11:11:46 +01:00
lwsl_parser("WSI_TOKEN_NAME_PART '%c' 0x%02X "
"(role=0x%lx) "
"wsi->lextable_pos=%d\n", c, c,
(unsigned long)lwsi_role(wsi),
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
ah->lextable_pos);
ah: workaround for broken routers with no http header delimiter There's a type of router in the wild issuing malformed http when in captive portal mode... there's no \x0a\x0d but just \x0a
2020-06-30 17:50:01 +01:00
if (!ah->unk_pos && c == '\x0a')
/* broken peer */
goto set_parsing_complete;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (c >= 'A' && c <= 'Z')
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
c = (unsigned char)(c + 'a' - 'A');
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
/*
* ...in case it's an unknown header, speculatively
* store it as the name comes in. If we recognize it as
* a known header, we'll snip this.
*/
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (!wsi->mux_substream && !ah->unk_pos) {
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
ah->unk_pos = ah->pos;
ah-un_pos-cleanup-fix-other-path
2020-06-30 17:49:14 +01:00
ah: use unk_pos to clean up unknown header starts even without CUSTOM_HEADERS
2020-06-26 07:34:27 +01:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
/*
* Prepare new unknown header linked-list entry
*
* - 16-bit BE: name part length
* - 16-bit BE: value part length
* - 32-bit BE: data offset of next, or 0
*/
for (n = 0; n < 8; n++)
if (!lws_pos_in_bounds(wsi))
ah->data[ah->pos++] = 0;
#endif
ah: use unk_pos to clean up unknown header starts even without CUSTOM_HEADERS
2020-06-26 07:34:27 +01:00
}
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
if (lws_pos_in_bounds(wsi))
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
return LPR_FAIL;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->data[ah->pos++] = (char)c;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
pos = ah->lextable_pos;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (!wsi->mux_substream && pos < 0 && c == ':') {
format strings: ban %.*s as some platforms lack it The %.*s is very handy to print strings where you have a length, but there is no NUL termination. It's quite widely supported but at least one vendor RTOS toolchain doesn't have it. Since there aren't that many uses of it yet, audit all uses and convert to a new helper lws_strnncpy() which uses the smaller of two lengths.
2019-11-04 10:54:50 +00:00
#if defined(_DEBUG)
char dotstar[64];
int uhlen;
#endif
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
/*
* process unknown headers
*
* register us in the unknown hdr ll
*/
if (!ah->unk_ll_head)
ah->unk_ll_head = ah->unk_pos;
if (ah->unk_ll_tail)
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
lws_ser_wu32be(
(uint8_t *)&ah->data[ah->unk_ll_tail + UHO_LL],
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
ah->unk_pos);
ah->unk_ll_tail = ah->unk_pos;
format strings: ban %.*s as some platforms lack it The %.*s is very handy to print strings where you have a length, but there is no NUL termination. It's quite widely supported but at least one vendor RTOS toolchain doesn't have it. Since there aren't that many uses of it yet, audit all uses and convert to a new helper lws_strnncpy() which uses the smaller of two lengths.
2019-11-04 10:54:50 +00:00
#if defined(_DEBUG)
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
uhlen = (int)(ah->pos - (ah->unk_pos + UHO_NAME));
format strings: ban %.*s as some platforms lack it The %.*s is very handy to print strings where you have a length, but there is no NUL termination. It's quite widely supported but at least one vendor RTOS toolchain doesn't have it. Since there aren't that many uses of it yet, audit all uses and convert to a new helper lws_strnncpy() which uses the smaller of two lengths.
2019-11-04 10:54:50 +00:00
lws_strnncpy(dotstar,
&ah->data[ah->unk_pos + UHO_NAME],
uhlen, sizeof(dotstar));
lwsl_debug("%s: unk header %d '%s'\n",
__func__,
ah->pos - (ah->unk_pos + UHO_NAME),
dotstar);
#endif
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
/* set the unknown header name part length */
parsers: use common unaligned accessors
2019-09-11 06:34:52 +01:00
lws_ser_wu16be((uint8_t *)&ah->data[ah->unk_pos],
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
(uint16_t)((ah->pos - ah->unk_pos) - UHO_NAME));
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
ah->unk_value_pos = ah->pos;
/*
* collect whatever's coming for the unknown header
* argument until the next CRLF
*/
ah->parser_state = WSI_TOKEN_UNKNOWN_VALUE_PART;
break;
}
#endif
if (pos < 0)
break;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
while (1) {
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
if (lextable_h1[pos] & (1 << 7)) {
clean: general whitespace cleanup
2018-11-23 08:47:56 +08:00
/* 1-byte, fail on mismatch */
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
if ((lextable_h1[pos] & 0x7f) != c) {
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
nope:
ah->lextable_pos = -1;
break;
}
/* fall thru */
pos++;
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
if (lextable_h1[pos] == FAIL_CHAR)
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
goto nope;
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->lextable_pos = (int16_t)pos;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
break;
}
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
if (lextable_h1[pos] == FAIL_CHAR)
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
goto nope;
/* b7 = 0, end or 3-byte */
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
if (lextable_h1[pos] < FAIL_CHAR) {
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (!wsi->mux_substream) {
h2: tolerate unexpected server behaviour Make sure we only do one SETTINGS ack handling Fix corner case of ignored header with 0 data being the last in the HEADERS
2019-10-10 16:37:38 +01:00
/*
* We hit a terminal marker, so
* we recognized this header...
* drop the speculative name
* part storage
*/
ah->pos = ah->unk_pos;
ah->unk_pos = 0;
}
ah: use unk_pos to clean up unknown header starts even without CUSTOM_HEADERS
2020-06-26 07:34:27 +01:00
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->lextable_pos = (int16_t)pos;
Add PATCH, PUT, DELETE methods AG: rewrite method code to use method arrays
2015-01-10 19:01:52 -08:00
break;
}
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
if (lextable_h1[pos] == c) { /* goto */
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->lextable_pos = (int16_t)(pos +
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
(lextable_h1[pos + 1]) +
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
(lextable_h1[pos + 2] << 8));
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
break;
}
/* fall thru goto */
pos += 3;
/* continue */
}
security harden http parser a bit Drop the connection during parsing for a few more cases that can't be legit. Take care about trying to free rxflow_buffer only if we reached a connmode where it exists Change behaviour on setting unknown HTTP method to kill connection Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-12 12:52:39 +08:00
/*
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
* If it's h1, server needs to be on the look out for
* unknown methods...
security harden http parser a bit Drop the connection during parsing for a few more cases that can't be legit. Take care about trying to free rxflow_buffer only if we reached a connmode where it exists Change behaviour on setting unknown HTTP method to kill connection Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-12 12:52:39 +08:00
*/
refactor role ops This only refactors internal architecture and representations, the user api is unaffected.
2018-04-11 13:39:42 +08:00
if (ah->lextable_pos < 0 && lwsi_role_h1(wsi) &&
lwsi_role_server(wsi)) {
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
/*
* this is not a header we know about... did
* we get a valid method (GET, POST etc)
* already, or is this the bogus method?
*/
libwebsockets.h: clean out some boilerplate better put in core/private.h https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
for (m = 0; m < LWS_ARRAY_SIZE(methods); m++)
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (ah->frag_index[methods[m]]) {
/*
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
* already had the method
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
*/
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#if !defined(LWS_WITH_CUSTOM_HEADERS)
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
ah->parser_state = WSI_TOKEN_SKIPPING;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#endif
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (wsi->mux_substream)
h2: tolerate unexpected server behaviour Make sure we only do one SETTINGS ack handling Fix corner case of ignored header with 0 data being the last in the HEADERS
2019-10-10 16:37:38 +01:00
ah->parser_state = WSI_TOKEN_SKIPPING;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
break;
}
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
h2: tolerate unexpected server behaviour Make sure we only do one SETTINGS ack handling Fix corner case of ignored header with 0 data being the last in the HEADERS
2019-10-10 16:37:38 +01:00
if (m != LWS_ARRAY_SIZE(methods)) {
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
/*
* We have the method, this is just an
* unknown header then
*/
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (!wsi->mux_substream)
h2: tolerate unexpected server behaviour Make sure we only do one SETTINGS ack handling Fix corner case of ignored header with 0 data being the last in the HEADERS
2019-10-10 16:37:38 +01:00
goto unknown_hdr;
else
break;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#else
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
break;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#endif
h2: tolerate unexpected server behaviour Make sure we only do one SETTINGS ack handling Fix corner case of ignored header with 0 data being the last in the HEADERS
2019-10-10 16:37:38 +01:00
}
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
/*
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
* ...it's an unknown http method from a client
* in fact, it cannot be valid http.
*
* Are we set up to transition to another role
* in these cases?
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
*/
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
if (lws_check_opt(wsi->a.vhost->options,
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
LWS_SERVER_OPTION_FALLBACK_TO_APPLY_LISTEN_ACCEPT_CONFIG)) {
lwsl_notice("%s: http fail fallback\n",
__func__);
/* transition to other role */
return LPR_DO_FALLBACK;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
}
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
lwsl_info("Unknown method - dropping\n");
goto forbid;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
}
if (ah->lextable_pos < 0) {
ah-un_pos-cleanup-fix-other-path
2020-06-30 17:49:14 +01:00
/*
* It's not a header that lws knows about...
*/
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (!wsi->mux_substream)
h2: tolerate unexpected server behaviour Make sure we only do one SETTINGS ack handling Fix corner case of ignored header with 0 data being the last in the HEADERS
2019-10-10 16:37:38 +01:00
goto unknown_hdr;
#endif
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
/*
* ...otherwise for a client, let him ignore
* unknown headers coming from the server
*/
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
ah->parser_state = WSI_TOKEN_SKIPPING;
break;
Subject: [PATCH] Fix bug parsing unknown headers. Bug was introduced in 49f72aa45.
2015-01-30 12:04:43 +00:00
}
parser restrict unknown header paranoia to server methods Signed-off-by: Andy Green <andy.green@linaro.org>
2015-04-07 08:19:30 +08:00
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
if (lextable_h1[ah->lextable_pos] < FAIL_CHAR) {
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* terminal state */
leverage TOKEN_SKIPPING better in parser Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-04 09:24:18 +08:00
rtos diet: make raw_file role optional
2020-02-28 15:50:15 +00:00
n = ((unsigned int)lextable_h1[ah->lextable_pos] << 8) |
lextable_h1[ah->lextable_pos + 1];
add lexical parser for headers Profiling what happens during the ab test, one of the hotspots was strcasecmp in a loop looking for header name matches each time. This patch introduces a lexical parser that creates a state machine in 276 bytes that encodes all the known header names. The fsm is walked bytewise as chaacters come in... most states do not need any recursion to match or fail. The state machine output is cut-and-pasted into parsers.c as an unsigned char array. The fsm generator is a bit rough and ready, included in the tree but not built since normal mortals won't need to touch it. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-01-18 01:55:48 +08:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
lwsl_parser("known hdr %d\n", n);
libwebsockets.h: clean out some boilerplate better put in core/private.h https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
for (m = 0; m < LWS_ARRAY_SIZE(methods); m++)
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (n == methods[m] &&
ah->frag_index[methods[m]]) {
lwsl_warn("Duplicated method\n");
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
return LPR_FAIL;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
}
security disallow repeated GET Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-12 13:10:19 +08:00
ah-un_pos-cleanup-fix-other-path
2020-06-30 17:49:14 +01:00
if (!wsi->mux_substream) {
/*
* Whether we are collecting unknown names or not,
* if we matched an internal header we can dispense
* with the header name part we were keeping
*/
ah->pos = ah->unk_pos;
ah->unk_pos = 0;
}
#if defined(LWS_ROLE_WS)
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/*
* WSORIGIN is protocol equiv to ORIGIN,
* JWebSocket likes to send it, map to ORIGIN
*/
if (n == WSI_TOKEN_SWORIGIN)
n = WSI_TOKEN_ORIGIN;
rtos diet: http: remove headers at buildtime according to config Headers related to ws or h2 are now elided if the ws or h2 role is not enabled for build. In addition, a new build-time option LWS_WITH_HTTP_UNCOMMON_HEADERS on by default allows removal of less-common http headers to shrink the parser footprint. Minilex is adapted to produce 8 different versions of the lex table, chosen at build-time according to which headers are included in the build. If you don't need the unusual headers, or aren't using h2 or ws, this chops down the size of the ah and the rodata needed to hold the parsing table from 87 strings / pointers to 49, and the parsing table from 1177 to 696 bytes.
2020-02-28 10:31:04 +00:00
#endif
map wsorigin on to origin at an early point and stop dupe header mem leaks Carlo wrote ''I'm interested to use the libwebsockets server (C based) with a Java websocket client. I was able to implement a Java websocket client based on JWebSocket library that works fine with the libwebsockets server, but I was obliged to patch the libwebsockets server in order to reach my goal. In particular I patched the handshake_00() function because JWebSocket library uses hixie 76 version. The problem was that the JWebSocket library sends only WSI_TOKEN_SWORIGIN token instead WSI_TOKEN_ORIGIN token. For this reason the handshake_00 failed because the token length of WSI_TOKEN_ORIGIN was 0. I have written a patch in order to control if at least one of the two tokens (WSI_TOKEN_SWORIGIN and WSI_TOKEN_ORIGIN) is defined. I have attached into the e-mail the patch. Probably it can be written better, but in any case I think that this problem should be fixed for client/server compatibility.'' I found a simpler approach which is convert WSORIGIN processing to go into ORIGIN token at the earliest point, then everything else can stay the same. I also noticed we would leak on duplicated headers and fixed that. Reported-by: Carlo PARATA <carlo.parata@st.com> Signed-off-by: Andy Green <andy@warmcat.com>
2012-04-03 17:02:20 +02:00
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->parser_state = (uint8_t)
clean: general whitespace cleanup
2018-11-23 08:47:56 +08:00
(WSI_TOKEN_GET_URI + n);
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
ah->ups = URIPS_IDLE;
Reducing CPU cycles spent calculating per-header length limit.
2014-06-29 01:34:24 -04:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (context->token_limits)
ah->current_token_limit = context->
clean: general whitespace cleanup
2018-11-23 08:47:56 +08:00
token_limits->token_limit[
ah->parser_state];
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
else
ah->current_token_limit =
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
wsi->a.context->max_http_header_data;
Reducing CPU cycles spent calculating per-header length limit.
2014-06-29 01:34:24 -04:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (ah->parser_state == WSI_TOKEN_CHALLENGE)
goto set_parsing_complete;
clean--fix-sigpipe.patch Signed-off-by: Andy Green <andy@warmcat.com>
2011-01-23 16:50:33 +00:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
goto start_fragment;
}
break;
make duplicate headers append instead of get ignored The code to prevent memory leaks caused by reallocating a wsi->utf8_token was also causing wsi->parser_state to not be updated, preventing the handshake from completing. This patch changes the logic to append to previous allocation which is correct behaviour actually. Signed-off-by: Nick Dowell <nick@nickdowell.com> Signed-off-by: Andy Green <andy@warmcat.com>
2012-04-05 10:31:48 +08:00
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#if defined(LWS_WITH_CUSTOM_HEADERS)
unknown_hdr:
//ah->parser_state = WSI_TOKEN_SKIPPING;
//break;
mux children: generalize helpers out of h2 implementation This should be a NOP for h2 support and only affects internal apis. But it lets us reuse the working and reliable h2 mux arrangements directly in other protocols later, and share code so building for h2 + new protocols can take advantage of common mux child handling struct and code. Break out common mux handling struct into its own type. Convert all uses of members that used to be in wsi->h2 to wsi->mux Audit all references to the members and break out generic helpers for anything that is useful for other mux-capable protocols to reuse wsi->mux related features.
2019-12-23 11:31:57 +00:00
if (!wsi->mux_substream)
h2: tolerate unexpected server behaviour Make sure we only do one SETTINGS ack handling Fix corner case of ignored header with 0 data being the last in the HEADERS
2019-10-10 16:37:38 +01:00
break;
ah: custom headers for h1 Until now lws only parses headers it knows at build-time from its prebuilt lexical analyzer. This adds an on-by-default cmake option and a couple of apis to also store and query "custom", ie, unknown-to-lws headers. A minimal example is also provided. At the moment it only works on h1, h2 support needs improvements to the hpack implementation. Since it bloats ah memory usage compared to without it if custom headers are present, the related code and ah footprint can be disabled with the cmake option LWS_WITH_CUSTOM_HEADERS, but it's on by default normally. ESP32 platform disables it. https://github.com/warmcat/libwebsockets/pull/1499
2019-02-26 17:18:24 +08:00
#endif
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
start_fragment:
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
ah->nfrag++;
http uri arguments process in fragments This makes the URI argument processing split each parameter into a "fragment". Processing header content as fragments already exists in lws, because it's legal to deliver header content by repeating the header. Now there's an api to access individual fragments, also add the code to the test server to print each URI argument separately. Adapt attack.sh to parse the fragments. Signed-off-by: Andy Green <andy.green@linaro.org>
2015-12-15 22:57:19 +08:00
excessive:
libwebsockets.h: clean out some boilerplate better put in core/private.h https://github.com/warmcat/libwebsockets/issues/1370
2018-08-16 19:10:32 +08:00
if (ah->nfrag == LWS_ARRAY_SIZE(ah->frags)) {
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
lwsl_warn("More hdr frags than we can deal with\n");
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
return LPR_FAIL;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
}
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
ah->frags[ah->nfrag].offset = ah->pos;
ah->frags[ah->nfrag].len = 0;
ah->frags[ah->nfrag].nfrag = 0;
ah->frags[ah->nfrag].flags = 2;
n = ah->frag_index[ah->parser_state];
if (!n) { /* first fragment */
ah->frag_index[ah->parser_state] = ah->nfrag;
ah->hdr_token_idx = ah->parser_state;
break;
}
/* continuation */
while (ah->frags[n].nfrag)
n = ah->frags[n].nfrag;
ah->frags[n].nfrag = ah->nfrag;
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (issue_char(wsi, ' ') < 0)
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
return LPR_FAIL;
improve minilex use external parsing header Clean up minilex Move the header output to stdout Introduce lexfile.h as the header output Use lexfile.h in both minilex itself and lws Add the following header support "Accept:", "If-Modified-Since:", "Accept-Encoding:", "Accept-Language:", "Pragma:", "Cache-Control:", "Authorization:", "Cookie:", "Content-Type:", "Date:", "Range:", "Referer:" Signed-off-by: Andy Green <andy.green@linaro.org>
2013-11-09 10:09:09 +08:00
break;
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
/* skipping arg part of a name we didn't recognize */
case WSI_TOKEN_SKIPPING:
lwsl_parser("WSI_TOKEN_SKIPPING '%c'\n", c);
coding-style-cleanups.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-13 10:03:47 +00:00
ah: workaround for broken routers with no http header delimiter There's a type of router in the wild issuing malformed http when in captive portal mode... there's no \x0a\x0d but just \x0a
2020-06-30 17:50:01 +01:00
if (c == '\x0a') {
/* broken peer */
ah->parser_state = WSI_TOKEN_NAME_PART;
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->unk_pos = 0;
ah->lextable_pos = 0;
ah: workaround for broken routers with no http header delimiter There's a type of router in the wild issuing malformed http when in captive portal mode... there's no \x0a\x0d but just \x0a
2020-06-30 17:50:01 +01:00
}
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
if (c == '\x0d')
ah->parser_state = WSI_TOKEN_SKIPPING_SAW_CR;
break;
Support for HTTP POST. Rewritten by Andy Green to use chunks, add timeout, use extra states in handshake not parser. Signed-off-by: Andy Green <andy@warmcat.com>
2013-11-19 13:38:16 +01:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
case WSI_TOKEN_SKIPPING_SAW_CR:
lwsl_parser("WSI_TOKEN_SKIPPING_SAW_CR '%c'\n", c);
if (ah->ues != URIES_IDLE)
goto forbid;
if (c == '\x0a') {
ah->parser_state = WSI_TOKEN_NAME_PART;
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
ah->unk_pos = 0;
ah->lextable_pos = 0;
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
} else
ah->parser_state = WSI_TOKEN_SKIPPING;
break;
/* we're done, ignore anything else */
simplify parsing complete Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-04 09:09:19 +08:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
case WSI_PARSING_COMPLETE:
lwsl_parser("WSI_PARSING_COMPLETE '%c'\n", c);
break;
}
Support for HTTP POST. Rewritten by Andy Green to use chunks, add timeout, use extra states in handshake not parser. Signed-off-by: Andy Green <andy@warmcat.com>
2013-11-19 13:38:16 +01:00
h1: header parsing optimizations
2018-03-07 18:15:17 +08:00
} while (*len);
coding-style-cleanups.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-13 10:03:47 +00:00
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
return LPR_OK;
simplify parsing complete Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-04 09:09:19 +08:00
set_parsing_complete:
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
if (ah->ues != URIES_IDLE)
urldecode forbid malformed And update attack.sh to confirm the new test cases Signed-off-by: Andy Green <andy@warmcat.com>
2016-04-07 10:08:35 +08:00
goto forbid;
threadpool
2018-09-02 14:35:37 +08:00
replace per header mallocs with single malloc 3 level struct This big patch replaces the malloc / realloc per header approach used until now with a single three-level struct that gets malloc'd during the header union phase and freed in one go when we transition to a different union phase. It's more expensive in that we malloc a bit more than 4Kbytes, but it's a lot cheaper in terms of malloc, frees, heap fragmentation, no reallocs, nothing to configure. It also moves from arrays of pointers (8 bytes on x86_64) to unsigned short offsets into the data array, (2 bytes on all platforms). The 3-level thing is all in one struct - array indexed by the header enum, pointing to first "fragment" index (ie, header type to fragment lookup, or 0 for none) - array of fragments indexes, enough for 2 x the number of known headers (fragment array... note that fragments can point to a "next" fragment if the same header is spread across multiple entries) - linear char array where the known header payload gets written (fragments point into null-terminated strings stored in here, only the known header content is stored) http headers can legally be split over multiple headers of the same name which should be concatenated. This scheme does not linearly conatenate them but uses a linked list in the fragment structs to link them. There are apis to get the total length and copy out a linear, concatenated version to a buffer. Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-10 18:02:31 +08:00
if (lws_hdr_total_length(wsi, WSI_TOKEN_UPGRADE)) {
rtos diet: http: remove headers at buildtime according to config Headers related to ws or h2 are now elided if the ws or h2 role is not enabled for build. In addition, a new build-time option LWS_WITH_HTTP_UNCOMMON_HEADERS on by default allows removal of less-common http headers to shrink the parser footprint. Minilex is adapted to produce 8 different versions of the lex table, chosen at build-time according to which headers are included in the build. If you don't need the unusual headers, or aren't using h2 or ws, this chops down the size of the ah and the rodata needed to hold the parsing table from 87 strings / pointers to 49, and the parsing table from 1177 to 696 bytes.
2020-02-28 10:31:04 +00:00
#if defined(LWS_ROLE_WS)
fanalyzer: workarounds and fixes GCC -fanalyzer did find a real issue (unchecked malloc return) but it mainly reported things that weren't true due to what was happening outside of the particular compilation unit that it could see.
2020-03-27 19:10:40 +00:00
const char *pv = lws_hdr_simple_ptr(wsi, WSI_TOKEN_VERSION);
if (pv)
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
wsi->rx_frame_type = (char)atoi(pv);
simplify parsing complete Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-04 09:09:19 +08:00
refactor-eliminate-lws-union The union used to make a lot of sense to save space between mutually exclusive modes. But the fact the http2 struct contains the http1 struct as well as it appearing in the union means the http1 struct belongs outside the union. This patch - eliminates the union - puts the http_related struct directly in struct lws - removes http_related from h2 - puts h2 directly in struct lws if enabled for build - changes ws to be a pointer, allocated if we upgrade to ws (the ws part contains a 135 byte char array for ping / close) Again all of this is entirely private / internal and doesn't affect any apis.
2017-12-01 11:09:32 +08:00
lwsl_parser("v%02d hdrs done\n", wsi->rx_frame_type);
rtos diet: http: remove headers at buildtime according to config Headers related to ws or h2 are now elided if the ws or h2 role is not enabled for build. In addition, a new build-time option LWS_WITH_HTTP_UNCOMMON_HEADERS on by default allows removal of less-common http headers to shrink the parser footprint. Minilex is adapted to produce 8 different versions of the lex table, chosen at build-time according to which headers are included in the build. If you don't need the unusual headers, or aren't using h2 or ws, this chops down the size of the ah and the rodata needed to hold the parsing table from 87 strings / pointers to 49, and the parsing table from 1177 to 696 bytes.
2020-02-28 10:31:04 +00:00
#endif
headers deleted after websocket established Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-06 15:15:25 +09:00
}
refactor-struct-lws-hdr Migrate what used to be in lws_hdr_related into either the ah or the wsi, and eliminate it along with the three different ways used to access things inside it. Eg wsi->u.hdr.ah and wsi->u.http.ah become wsi->ah These changes are internal-only, in private-libwebsockets.h and lib.
2017-11-16 11:26:00 +08:00
ah->parser_state = WSI_PARSING_COMPLETE;
fix ssl reject path for freeing header allocation If the SSL connection failed before the headers came, we were not dealing with deallocating the header malloc. This takes care of it. Using CyaSSL, we are then valgrind-clean for ssl client and server. With OpenSSL, there is 88 bytes lost at init that never changes or gets recovered. AFAIK there's nothing to do about that. OpenSSL also blows these during operation ==1059== Conditional jump or move depends on uninitialised value(s) ==1059== at 0x4A0B131: bcmp (mc_replace_strmem.c:935) ==1059== by 0x3014CDDBA8: ??? (in /usr/lib64/libcrypto.so.1.0.1c) ==1059== by 0x3015430852: tls1_enc (in /usr/lib64/libssl.so.1.0.1c) ==1059== by 0x3015428CEC: ssl3_read_bytes (in /usr/lib64/libssl.so.1.0.1c) ==1059== by 0x30154264C5: ??? (in /usr/lib64/libssl.so.1.0.1c) ==1059== by 0x4C3C596: lws_server_socket_service (server.c:153) ==1059== by 0x4C32C1E: libwebsocket_service_fd (libwebsockets.c:927) ==1059== by 0x4C33270: libwebsocket_service (libwebsockets.c:1225) ==1059== by 0x401C84: main (in /usr/bin/libwebsockets-test-server) However googling around https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/60021 http://www.openssl.org/support/faq.html#PROG13 (also the next FAQ down) it seems OpenSSL have a relaxed attitude to this and it's expected. It's interesting CyaSSL works fine but doesn't have that problem... Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-11 21:43:41 +08:00
wsi->hdr_parsing_completed = 1;
simplify parsing complete Signed-off-by: Andy Green <andy.green@linaro.org>
2013-02-04 09:09:19 +08:00
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
return LPR_OK;
urldecode forbid malformed And update attack.sh to confirm the new test cases Signed-off-by: Andy Green <andy@warmcat.com>
2016-04-07 10:08:35 +08:00
forbid:
minimal-http-client-multi: add POST This adds support for POST in both h1 and h2 queues / stream binding. The previous queueing tried to keep the "leader" wsi who made the actual connection around and have it act on the transaction queue tail if it had done its own thing. This refactors it so instead, who is the "leader" moves down the queue and the queued guys inherit the fd, SSL * and queue from the old leader as they take over. This lets them operate in their own wsi identity directly and gets rid of all the "effective wsi" checks, which was applied incompletely and getting out of hand considering the separate lws_mux checks for h2 and other muxed protocols alongside it. This change also allows one wsi at a time to own the transaction for POST. --post is added as an option to lws-minimal-http-client-multi and 6 extra selftests with POST on h1/h2, pipelined or not and staggered or not are added to the CI.
2020-01-30 13:19:11 +00:00
lwsl_info(" forbidding on uri sanitation\n");
clean: internally use LWS_WITH_CLIENT and _SERVER Remove some more things in LWS_WITH_SERVER=0 case
2019-08-18 05:04:15 +01:00
#if defined(LWS_WITH_SERVER)
urldecode forbid malformed And update attack.sh to confirm the new test cases Signed-off-by: Andy Green <andy@warmcat.com>
2016-04-07 10:08:35 +08:00
lws_return_http_status(wsi, HTTP_STATUS_FORBIDDEN, NULL);
clean: internally use LWS_WITH_CLIENT and _SERVER Remove some more things in LWS_WITH_SERVER=0 case
2019-08-18 05:04:15 +01:00
#endif
http2: make usable HTTP/2 support is now able to serve the test server, complete with websockets, from a single vhost. - This works the same with both OpenSSL and mbedTLS. - POST is now wired up and works (also for file upload). - CGI is wired up and works. - Redirect is adapted and works - lwsws works. - URI urldecode, sanitation and argument parsing wired up for :path valgrind clean (aside from openssl-style false uninit data usage in mbedtls send occasionally) h2spec reports: $ h2spec -h 127.0.0.1 -p 7681 -t -k -o 1 ... 145 tests, 145 passed, 0 skipped, 0 failed" Incorporates: - "https://github.com/warmcat/libwebsockets/pull/1039 Fixes issue with -Werror=unused-variable flag - https://github.com/sartura/libwebsockets/commit/2c843a1395bdca5c23b40d989f508fc96aa5ba3b ssl: fix infinite loop on client cert verification failure Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>" Caused and fixes Coverity 184887 - 184892
2017-10-13 10:33:02 +08:00
adopt: allow associated accepted vhost connections to specific role Normalize the vhost options around optionally handling noncompliant traffic at the listening socket for both non-tls and tls cases. By default everything is as before. However it's now possible to tell the vhost to allow noncompliant connects to fall back to a specific role and protocol, both set by name in the vhost creation info struct. The original vhost flags allowing http redirect to https and direct http serving from https server (which is a security downgrade if enabled) are cleaned up and tested. A minimal example minimal-raw-fallback-http-server is added with switches to confirm operation of all the valid possibilities (see the readme on that).
2018-11-29 08:47:49 +08:00
return LPR_FORBIDDEN;
break-out-lib-sources.patch Signed-off-by: Andy Green <andy@warmcat.com>
2010-11-08 20:20:42 +00:00
}
lws_http_cookie_get
2020-07-10 12:48:10 +01:00
int
lws_http_cookie_get(struct lws *wsi, const char *name, char *buf,
size_t *max_len)
{
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
size_t max = *max_len, bl = strlen(name);
lws_http_cookie_get
2020-07-10 12:48:10 +01:00
char *p, *bo = buf;
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
int n;
lws_http_cookie_get
2020-07-10 12:48:10 +01:00
n = lws_hdr_total_length(wsi, WSI_TOKEN_HTTP_COOKIE);
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
if ((unsigned int)n < bl + 1)
lws_http_cookie_get
2020-07-10 12:48:10 +01:00
return 1;
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
/*
* This can come to us two ways, in ah fragments (h2) or as a single
* semicolon-delimited string (h1)
*/
#if defined(LWS_ROLE_H2)
if (lws_hdr_total_length(wsi, WSI_TOKEN_HTTP_COLON_METHOD)) {
/*
* The h2 way...
*/
int f = wsi->http.ah->frag_index[WSI_TOKEN_HTTP_COOKIE];
size_t fl;
while (f) {
p = wsi->http.ah->data + wsi->http.ah->frags[f].offset;
fl = (size_t)wsi->http.ah->frags[f].len;
if (fl >= bl + 1 &&
p[bl] == '=' &&
!memcmp(p, name, bl)) {
fl -= bl + 1;
if (max - 1 < fl)
fl = max - 1;
if (fl)
memcpy(buf, p + bl + 1, fl);
*max_len = fl;
buf[fl] = '\0';
return 0;
}
f = wsi->http.ah->frags[f].nfrag;
}
return -1;
}
#endif
/*
* The h1 way...
*/
lws_http_cookie_get
2020-07-10 12:48:10 +01:00
p = lws_hdr_simple_ptr(wsi, WSI_TOKEN_HTTP_COOKIE);
if (!p)
return 1;
p += bl;
set-cookie: force semicolon for concatenation
2021-01-19 20:45:31 +00:00
n -= (int)bl;
http: cookie_get The loop check shouldn't care about bl any more, since we snipped bl from n already.
2021-12-02 09:18:23 +00:00
while (n-- > 0) {
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
if (*p == '=' && !memcmp(p - bl, name, (unsigned int)bl)) {
lws_http_cookie_get
2020-07-10 12:48:10 +01:00
p++;
while (*p != ';' && n-- && max) {
*buf++ = *p++;
max--;
}
if (!max)
return 2;
*buf = '\0';
type comparisons: fixes This is a huge patch that should be a global NOP. For unix type platforms it enables -Wconversion to issue warnings (-> error) for all automatic casts that seem less than ideal but are normally concealed by the toolchain. This is things like passing an int to a size_t argument. Once enabled, I went through all args on my default build (which build most things) and tried to make the removed default cast explicit. With that approach it neither change nor bloat the code, since it compiles to whatever it was doing before, just with the casts made explicit... in a few cases I changed some length args from int to size_t but largely left the causes alone. From now on, new code that is relying on less than ideal casting will complain and nudge me to improve it by warnings.
2020-12-12 06:21:40 +00:00
*max_len = lws_ptr_diff_size_t(buf, bo);
lws_http_cookie_get
2020-07-10 12:48:10 +01:00
return 0;
}
p++;
}
return 1;
}
lws_jwt_token_sanity
2020-07-11 07:16:16 +01:00
#if defined(LWS_WITH_JOSE)
#define MAX_JWT_SIZE 1024
int
lws_jwt_get_http_cookie_validate_jwt(struct lws *wsi,
struct lws_jwt_sign_set_cookie *i,
char *out, size_t *out_len)
{
char temp[MAX_JWT_SIZE * 2];
size_t cml = *out_len;
const char *cp;
/* first use out to hold the encoded JWT */
if (lws_http_cookie_get(wsi, i->cookie_name, out, out_len)) {
reduce debug logging
2020-11-06 20:25:36 +00:00
lwsl_debug("%s: cookie %s not provided\n", __func__,
lws_jwt_token_sanity
2020-07-11 07:16:16 +01:00
i->cookie_name);
return 1;
}
/* decode the JWT into temp */
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
if (lws_jwt_signed_validate(wsi->a.context, i->jwk, i->alg, out,
lws_jwt_token_sanity
2020-07-11 07:16:16 +01:00
*out_len, temp, sizeof(temp), out, &cml)) {
reduce debug logging
2020-11-06 20:25:36 +00:00
lwsl_info("%s: jwt validation failed\n", __func__);
lws_jwt_token_sanity
2020-07-11 07:16:16 +01:00
return 1;
}
/*
* Copy out the decoded JWT payload into out, overwriting the
* original encoded JWT taken from the cookie (that has long ago been
* translated into allocated buffers in the JOSE object)
*/
if (lws_jwt_token_sanity(out, cml, i->iss, i->aud, i->csrf_in,
i->sub, sizeof(i->sub),
&i->expiry_unix_time)) {
lwsl_notice("%s: jwt sanity failed\n", __func__);
return 1;
}
/*
* If he's interested in his private JSON part, point him to that in
* the args struct (it's pointing to the data in out
*/
cp = lws_json_simple_find(out, cml, "\"ext\":", &i->extra_json_len);
if (cp)
i->extra_json = cp;
reduce debug logging
2020-11-06 20:25:36 +00:00
if (!cp)
lws_jwt_token_sanity
2020-07-11 07:16:16 +01:00
lwsl_notice("%s: no ext JWT payload\n", __func__);
return 0;
}
int
lws_jwt_sign_token_set_http_cookie(struct lws *wsi,
const struct lws_jwt_sign_set_cookie *i,
uint8_t **p, uint8_t *end)
{
char plain[MAX_JWT_SIZE + 1], temp[MAX_JWT_SIZE * 2], csrf[17];
size_t pl = sizeof(plain);
unsigned long long ull;
int n;
/*
* Create a 16-char random csrf token with the same lifetime as the JWT
*/
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
lws_hex_random(wsi->a.context, csrf, sizeof(csrf));
lws_jwt_token_sanity
2020-07-11 07:16:16 +01:00
ull = lws_now_secs();
fakewsi: replace with smaller substructure Currently we always reserve a fakewsi per pt so events that don't have a related actual wsi, like vhost-protocol-init or vhost cert init via protocol callback can make callbacks that look reasonable to user protocol handler code expecting a valid wsi every time. This patch splits out stuff that user callbacks often unconditionally expect to be in a wsi, like context pointer, vhost pointer etc into a substructure, which is composed into struct lws at the top of it. Internal references (struct lws is opaque, so there are only internal references) are all updated to go via the substructre, the compiler should make that a NOP. Helpers are added when fakewsi is used and referenced. If not PLAT_FREERTOS, we continue to provide a full fakewsi in the pt as before, although the helpers improve consistency by zeroing down the substructure. There is a huge amount of user code out there over the last 10 years that did not always have the minimal examples to follow, some of it does some unexpected things. If it is PLAT_FREERTOS, that is a newer thing in lws and users have the benefit of being able to follow the minimal examples' approach. For PLAT_FREERTOS we don't reserve the fakewsi in the pt any more, saving around 800 bytes. The helpers then create a struct lws_a (the substructure) on the stack, zero it down (but it is only like 4 pointers) and prepare it with whatever we know like the context. Then we cast it to a struct lws * and use it in the user protocol handler call. In this case, the remainder of the struct lws is undefined. However the amount of old protocol handlers that might touch things outside of the substructure in PLAT_FREERTOS is very limited compared to legacy lws user code and the saving is significant on constrained devices. User handlers should not be touching everything in a wsi every time anyway, there are several cases where there is no valid wsi to do the call with. Dereference of things outside the substructure should only happen when the callback reason shows there is a valid wsi bound to the activity (as in all the minimal examples).
2020-07-19 08:33:46 +01:00
if (lws_jwt_sign_compact(wsi->a.context, i->jwk, i->alg, plain, &pl,
lws_jwt_token_sanity
2020-07-11 07:16:16 +01:00
temp, sizeof(temp),
"{\"iss\":\"%s\",\"aud\":\"%s\","
"\"iat\":%llu,\"nbf\":%llu,\"exp\":%llu,"
"\"csrf\":\"%s\",\"sub\":\"%s\"%s%s%s}",
i->iss, i->aud, ull, ull - 60,
ull + i->expiry_unix_time,
csrf, i->sub,
i->extra_json ? ",\"ext\":{" : "",
i->extra_json ? i->extra_json : "",
i->extra_json ? "}" : "")) {
lwsl_err("%s: failed to create JWT\n", __func__);
return 1;
}
/*
* There's no point the browser holding on to a JWT beyond the JWT's
* expiry time, so set it to be the same.
*/
n = lws_snprintf(temp, sizeof(temp), "__Host-%s=%s;"
"HttpOnly;"
"Secure;"
"SameSite=strict;"
"Path=/;"
"Max-Age=%lu",
i->cookie_name, plain, i->expiry_unix_time);
if (lws_add_http_header_by_token(wsi, WSI_TOKEN_HTTP_SET_COOKIE,
(uint8_t *)temp, n, p, end)) {
lwsl_err("%s: failed to add JWT cookie header\n", __func__);
return 1;
}
return 0;
}
#endif
Reference in a new issue Copy permalink