https://github.com/warmcat/libwebsockets/issues/489
This
1) fixes the vhost changes on master
2) works around the ah pool changes
3) fixes some other build problems that appeared
4) hacks out physical flow control for internal streams
5) updates the advertised protocol to h2 needed by, eg, chrome 51
That gets it able to serve small (<4K, ie, one packet) files over http2
Signed-off-by: Andy Green <andy@warmcat.com>
After discussion here
https://libwebsockets.org/pipermail/libwebsockets/2016-April/002268.html
scandir usage in
- lwsws conf.c
- lws plugin support
and
- lws plugin apis for dl
are converted to us libuv apis so they can work cross-platform easily.
lws itself remains not requiring libuv, although it's an option.
- LWS_WITH_LWSWS
- LWS_WITH_PLUGINS
now force LWS_WITH_LIBUV if selected... both of these are new features
only in master atm and both are off by default in CMake.
There's a complication libuv can be too old to offer the necessary apis,
this is the case in Travis Trusty instance. In that case, UV_VERSION_MAJOR ==0,
then the unix-only plugin implementation is used instead.
Signed-off-by: Andy Green <andy@warmcat.com>
This adds support for dynamically loaded plugins at runtime, which
can expose their own protocols or extensions transparently.
With these changes lwsws defaults to OFF in cmake, and if enabled it
automatically enables plugins and libuv support.
Signed-off-by: Andy Green <andy@warmcat.com>
Fix building libwebsockets with the musl C libary.
<sys/cdefs.h> is an internal glibc header and should be avoided in user code.
__P() was used for compatibility with some old K&R C compilers, when there were
no prototypes (which were introduced to C with C89). As supporting legacy
non-ANSI compilers is nowadays not necessary anymore get rid of the unnecessary
function prototype using __P().
polarssl is the old name for mbedtls. Unfortunately the two are confused in eg,
Fedora. For our purposes, polarssl or mbedtls < 2.0 has includes in
/usr/include/polarssl and polarssl_ apis and we call that "polarssl".
polarssl or mbedtls >=2.0 has includes in /usr/include/mbedtls and mbedtls_ apis,
we call that "mbedtls".
This has to be spelled out clearly because eg Fedora has a package "mbedtls" which
is 1.3.x and has the polarssl_ apis and include path. We will deal with that as
"polarssl" despite the package name then.
This patch lets you use LWS_USE_POLARSSL or LWS_USE_MBEDTLS and set the include and
library path like this
cmake .. -DLWS_USE_POLARSSL=1 -DLWS_POLARSSL_INCLUDE_DIRS=/usr/include -DLWS_POLARSSL_LIBRARIES=/usr/lib64/libmbedtls.so.9
This patch adds the cmake support and adapts [private-]libwebsockets.h but doesn't
modify the apis in ssl[-*].c yet.
Signed-off-by: Andy Green <andy@warmcat.com>
This patch splits out some lws_context members into a new lws_vhost struct.
- ssl state and options per vhost
- SSL_CTX for serving and client per vhost
- protocols[] per vhost
- extensions[] per vhost
lws_context maintains a linked list of lws_vhosts.
The same lws_context_creation_info struct is used to regulate both the
context creation and to create vhosts: for backward compatibility if you
didn't provide the new LWS_SERVER_OPTION_EXPLICIT_VHOSTS option, then
a default vhost is created at context creation time using the same info
data as the context itself.
If you will have multiple vhosts though, you should give the
LWS_SERVER_OPTION_EXPLICIT_VHOSTS option at context creation time,
create the context first and then the vhosts afterwards using
lws_create_vhost(contest, &info);
Although there is a lot of housekeeping to implement this change, there
is almost no additional overhead if you don't use multiple vhosts and
very little api impact (no changes to test apps).
Signed-off-by: Andy Green <andy@warmcat.com>
https://github.com/warmcat/libwebsockets/issues/468
Adds lws_check_opt() to regularize multibit flag checking.
There's a new context creation flag LWS_SERVER_OPTION_DO_SSL_GLOBAL_INIT,
this is included automatically if you give any other SSL-related option flag.
If you give no SSL-related option flag, nor this one directly, then even
though SSL support may be compiled in, it is never initialized nor used for the
whole lifetime of the lws context.
Conversely in order to prepare the context to use SSL, even though, eg, you
are not listening on SSL but will use SSL client connections later, you can
give this flag explicitly to make sure SSL is initialized.
Signed-off-by: Andy Green <andy@warmcat.com>
If you enable -DLWS_WITH_HTTP_PROXY=1 at cmake, the test server has a
new URI path http://localhost:7681/proxytest If you visit here, a client
connection to http://example.com:80 is spawned, and the results piped on
to your original connection.
Also with LWS_WITH_HTTP_PROXY enabled at cmake, lws wants to link to an
additional library, "libhubbub". This allows lws to do html rewriting on the
fly, adjusting proxied urls in a lightweight and fast way.
Move the socket bind to interface code out of server into
libwebsockets.c and make a private api for it.
Signed-off-by: Andy Green <andy.green@linaro.org>
wsi can have a full tree relationship with each other using
linked lists. closing the parent ensures the children are
closed first.
Convert cgi to use this instead of his cgi-specific sub-wsi
management.
Signed-off-by: Andy Green <andy.green@linaro.org>
Server support for http[s] as well as ws[s] is implicit.
But until now client only supported ws[s].
This allows the user code to pass an explicit http method
like "GET" in the connect_info, disabling the ws upgrade logic.
Then you can also use lws client as http client, not just ws.
Signed-off-by: Andy Green <andy.green@linaro.org>
It can join the free ah list and pick up client connect processing
later when the ah becomes available; this simplifies the code
doing the request since he won't have to deal with unexpected
failures / retries based on dynamic ah availability.
To do this though we have to handle that the connect_info members
may not have scope that lets them still exist after we return from
the first connect call, we stash them in a malloc'd buffer so the
connect processing can have them much later even so.
Signed-off-by: Andy Green <andy.green@linaro.org>
Originally this was alright in wsi->u.hdr, because ah implied header
processing. But since we allowed ah to be held across http
keep-alive transactions if we saw we had more header data, it means
we were trying to read this union member out of scope after it had
transitioned.
Moving the more_rx_waiting member to be a 1-bit bifield in the wsi
solves it and lets us check the state any time later at http
transaction completion.
https://github.com/warmcat/libwebsockets/issues/441
Signed-off-by: Andy Green <andy.green@linaro.org>
This adds an info member that allows the user code to
set the library's network action timeout in seconds.
If left at the default 0, the build-time default
AWAITING_TIMEOUT continues to be used.
As suggested
https://github.com/warmcat/libwebsockets/issues/427
Signed-off-by: Andy Green <andy.green@linaro.org>
This is intended to solve a longstanding problem with the
relationship between http/1.1 keep-alive and the service
loop.
Ah now contain an rx buffer which is used during header
processing, and the ah may not be detached from the wsi
until the rx buffer is exhausted.
Having the rx buffer in the ah means we can delay using the
rx until a later service loop.
Ah which have pending rx force POLLIN service on the wsi
they are attached to automatically, so we can interleave
general service / connections with draining each ah rx
buffer.
The possible http/1.1 situations and their dispositions are:
1) exactly one set of http headers come. After processing,
the ah is detached since no pending rx left. If more
headers come later, a fresh ah is aqcuired when available
and the rx flow control blocks the read until then.
2) more that one whole set of headers come and we remain in
http mode (no upgrade). The ah is left attached and
returns to the service loop after the first set of headers.
We will get forced service due to the ah having pending
content (respecting flowcontrol) and process the pending
rx in the ah. If we use it all up, we will detach the
ah.
3) one set of http headers come with ws traffic appended.
We service the headers, do the upgrade, and keep the ah
until the remaining ws content is used. When we
exhausted the ws traffix in the ah rx buffer, we
detach the ah.
Since there can be any amount of http/1.1 pipelining on a
connection, and each may be expensive to service, it's now
enforced there is a return to the service loop after each
header set is serviced on a connection.
When I added the forced service for ah with pending buffering,
I added support for it to the windows plat code. However this
is untested.
Signed-off-by: Andy Green <andy.green@linaro.org>
This gets the libuv stuff plumbed in and working.
Currently it's only workable for some service thread, and there
is an isolated valgrind problem left
==28425== 128 bytes in 1 blocks are definitely lost in loss record 3 of 3
==28425== at 0x4C28C50: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==28425== by 0x4C2AB1E: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==28425== by 0x58BBB27: maybe_resize (core.c:748)
==28425== by 0x58BBB27: uv__io_start (core.c:787)
==28425== by 0x58C1B80: uv__signal_loop_once_init (signal.c:225)
==28425== by 0x58C1B80: uv_signal_init (signal.c:260)
==28425== by 0x58BF7A6: uv_loop_init (loop.c:66)
==28425== by 0x4157F5: lws_uv_initloop (libuv.c:89)
==28425== by 0x405536: main (test-server-libuv.c:284)
libuv wants to sign off on all libuv 'handles' that will close, and
callback to do the close confirmation asynchronously. The wsi close function
is adapted when libuv is in use to work with libuv accordingly and exit the uv
loop the number of remaining wsi is zero.
Signed-off-by: Andy Green <andy.green@linaro.org>
Connections must hold an ah for the whole time they are
processing one header set, even if eg, the headers are
fragmented and it involves network roundtrip times.
However on http1.1 / keepalive, it must drop the ah when
there are no more header sets to deal with, and reacquire
the ah later when more data appears. It's because the
time between header sets / http1.1 requests is unbounded
and the ah would be tied up forever.
But in the case that we got pipelined http1.1 requests,
even partial already buffered, we must keep the ah,
resetting it instead of dropping it. Because we store
the rx data conveniently in a per-tsi buffer since it only
does one thing at a time per thread, we cannot go back to
the event loop to await a new ah inside one service action.
But no problem since we definitely already have an ah,
let's just reuse it at http completion time if more rx is
already buffered.
NB: attack.sh makes request with echo | nc, this
accidentally sends a trailing '\n' from the echo showing
this problem. With this patch attack.sh can complete well.
Signed-off-by: Andy Green <andy.green@linaro.org>
This adds support for multithreaded service to lws without adding any
threading or locking code in the library.
At context creation time you can request split the service part of the
context into n service domains, which are load-balanced so that the most
idle one gets the next listen socket accept.
There's a single listen socket on one port still.
User code may then spawn n threads doing n service loops / poll()s
simultaneously. Locking is only required (I think) in the existing
FD lock callbacks already handled by the pthreads server example,
and that locking takes place in user code. So the library remains
completely agnostic about the threading / locking scheme.
And by default, it's completely compatible with one service thread
so no changes are required by people uninterested in multithreaded
service.
However for people interested in extremely lightweight mass http[s]/
ws[s] service with minimum provisioning, the library can now do
everything out of the box.
To test it, just try
$ libwebsockets-test-server-pthreads -j 8
where -j controls the number of service threads
Signed-off-by: Andy Green <andy.green@linaro.org>
In the case we have a lot of connections, checking them all for timeout state
once a second becomes burdensome. At the moment if you have 100K connections,
once a second they all get checked for timeout in a loop.
This patch adds a doubly-linked list based in the context to each wsi, and
only wsi with pending timeouts appear on it. At checking time, we traverse
the list, which costs nothing if empty because nobody has a pending timeout.
Similarly adding and removing from the list costs almost nothing since no
iteration is required no matter how big the list.
The extra 8 or 16 bytes in the wsi are offset a little bit by demoting .pps
from int to char (save 3 bytes). And trim max act exts to 2, since we only
provide one, saving 8 /16 bytes by itself if exts enabled.
Signed-off-by: Andy Green <andy.green@linaro.org>
This adds redirect support to the client side. Lws will follow
server redirects (301) up to three deep.
Signed-off-by: Andy Green <andy.green@linaro.org>
Remove declarations of callback and extension_callback as these are
functions declared in header but not defined anywhere.
Also rename typedefs callback_function and extension_callback_function
to lws_callback_function and lws_extension_callback_function so all
symbolx exported by header have lws prefix;
Signed-off-by: Denis Osvald <denis.osvald@sartura.hr>
Considering we go through it once per incoming char, the tests to see if we
should be checking utf-8 are too expensive... move them to a bit that lives
in the wsi and set them once per frame (except for CLOSE who has to update
after the close code has been skipped).
Signed-off-by: Andy Green <andy.green@linaro.org>
Server side has had immediate RX flow control for quite a while.
But client side made do with RX continuing until what had been received was exhausted.
For what Autobahn tests, that's not enough.
This patch gives clientside RX flow control the same immediate effect as the server
side enjoys, re-using the same code.
Signed-off-by: Andy Green <andy.green@linaro.org>
