Large chunks of struct libwebsocket members actually have a mutually
exclusive lifecycle, eg, once the http headers are finished they sit
there unused until the instance is destroyed.
This makes a big improvement in memory efficiency by making four
categories of member: always needed, needed for header processing,
needed for http processing, and needed for ws processing. The last
three are mutually exclusive and bound into a union inside the wsi.
Care needs taking now at "union transitions", although we zeroed down
the struct at init, the other union siblings have been writing the
same memory by the time later member siblings start to use it. So
it must be cleared down appropriately when we cross from one
mutually-exclusive use to another.
Signed-off-by: Andy Green <andy.green@linaro.org>
Since v13 was defined as the released ietf version the older versions
are deprecated. This patch strips out everything to do with the older
versions and gets rid of the option to send stuff unmasked.
The in-tree md5 implementation is then also deleted as nothing needs
it any more, 1280 loc are shed in all
Signed-off-by: Andy Green <andy.green@linaro.org>
The whole thing about count_protocols + 1 broadcast sockets and
associated dummy wsis is a workaround for getting a broadcast from
a different process context, if we are running with --enable-no-fork
then we don't need any of it in.
Signed-off-by: Andy Green <andy.green@linaro.org>
The new --without-extensions config flag completely removes all code
and data related to extensions from the build throughout the library
when given.
Signed-off-by: Andy Green <andy.green@linaro.org>
Move server-only stuff into their own files and make building
that depend on not having --without-server on the configure
Make fragments in other places conditional as well
Remove client-related members from struct libwebscket when
building LWS_NO_CLIENT
Apps:
normal: build test server, client, fraggle, ping
--without-client: build test server
--without-server: build test client, ping
Signed-off-by: Andy Green <andy.green@linaro.org>
Profiling what happens during the ab test, one of the hotspots
was strcasecmp in a loop looking for header name matches each time.
This patch introduces a lexical parser that creates a state machine
in 276 bytes that encodes all the known header names. The fsm is
walked bytewise as chaacters come in... most states do not need any
recursion to match or fail.
The state machine output is cut-and-pasted into parsers.c as an
unsigned char array.
The fsm generator is a bit rough and ready, included in the tree but
not built since normal mortals won't need to touch it.
Signed-off-by: Andy Green <andy.green@linaro.org>
Problems with rx flow control implementation were the underlying cause
of the connection stalling issue that was covered up with the udelay()
patch that was removed recently.
This get rx flow control working properly and corrects problems with
fifo management in the test server mirror protocol code too.
The rxfow control api has been changed to just set a flag, so it's very cheap
to call from user code. After the callbacks that might use the rxflow control
api the flag is checked and any pending actions done.
rx flow control now stops any rx packet coming immediately, with compessed
connections "just what was left in the pipe" might be hundreds of KBytes. To
implement that the current packet being decoded is copied into a malloc'd buffer
by the rx processing code now.
When rxflow is allows to come again, the buffer is drained and freed before any
new packet content is accepted.
Signed-off-by: Andy Green <andy.green@linaro.org>
This rips out the connection hashtable implementation along with
MAX_CLIENTS and replaces it with a dynamically allocated fds array
and lookup table along the same lines as the new extpoll implementation
from Edwin van den Oetelaar.
It detects the max number of file descriptors possible at context init
time and allocates accordingly; this can be externally controlled by
ulimit and the server run as a specific user to facilitate targeting
specific ulimit rules at it.
Many operations that translated between socket descriptors and struct
websocket or pollfd objects have had iteration removed by this patch
and under load will be a lot faster.
Signed-off-by: Andy Green <andy.green@linaro.org>
From an idea by Edwin van den Oetelaar <oetelaar.automatisering@gmail.com>
When testing libwebsockets with ab, Edwin found an unexpected bump in
the distribution of latencies, some connections were held back almost
the whole test duration.
http://ml.libwebsockets.org/pipermail/libwebsockets/2013-January/000006.html
Studying the problem revealed that when there are mass pending connections
amongst many active connections, we do not service the listen socket often
enough to clear the backlog, some seem to get stale violating FIFO ordering.
This patch introduces listen socket service "piggybacking", where every n
normal socket service actions we also check the listen socket and deal with
pending connections there.
Normally, it checks the listen socket gratuitously every 10 normal socket
services. However, if it finds something waiting, it forces a check on the
next normal socket service too by keeping stats on how often something was
waiting. If the probability of something waiting each time becomes high,
it will allow up to two waiting connections to be serviced for each normal
socket service.
In that way it has low burden in the normal case, but rapidly adapts by
detecting mass connection loads as found in ab.
Signed-off-by: Andy Green <andy.green@linaro.org>
Default remains at SOMAXCONN, you can force it at configure time
along these lines
./configure CFLAGS="-DLWS_SOMAXCONN=16384"
Signed-off-by: Andy Green <andy.green@linaro.org>
Previously we sat and looped to dump a file over http protocol.
Actually that's a source of blocking to the other sockets being serviced.
This patch breaks up the file service into a roundtrip around the poll()
loop for each 512-byte packet. It doesn't make much difference if the
server is idle, but if it's busy it makes sure everyone else is getting
service while the file is sent.
It doesn't try to optimize multiple users of the file or to keep the
descriptor open, the point of this patch is to establish the breaking up
of the file send action into the poll loop.
On the user side, there are two differences:
- context is now needed in the first argument to libwebsockets_serve_http_file()
that's not too bad since we provide context in the callback.
- file send is now asynchronous to the user code, you get a new callback coming
in protocol 0 when it's done, LWS_CALLBACK_HTTP_FILE_COMPLETION
libwebsockets-test-server is updated accordingly.
Signed-off-by: Andy Green <andy.green@linaro.org>
"4b0e01f Retry SSL_connect when SSL_get_error requests it. " from David Galeano
noticed the problem that client connect may receive SSL_ERROR_WANT_* from
SSL_connect, which is basically WOULDBLOCK. That patch tried to deal with it
by blocking in a while(1) until the condition went away.
That's problematic because of it blocks service of anything else (including
the host application sockets in the external socket poll sharing case) for
up to 5s controlled by conditions at one client.
After fiddling with and researching this, the actual problem with the code is
we are not getting the SSL layer error correctly, it is not contained in the
code returned from the Connect api directly.
I was unable to get a renegotiation forced on my modern SSL libs, it complained
about protocol error are reopened the connection instead. So I think the stuff
found in the docs and the web about the SSL_ERROR_WANT_ is probably not something
we will see in reality (if we check the right error code...)
Signed-off-by: Andy Green <andy.green@linaro.org>
This patch allows control of the main compiletime constants in libwebsockets
from the configure commandline.
README is updated with documentation on what's available, how to set them
and the defaults.
The constants are logged with "info" severity (not visible by default) at
context create time.
The zlib constant previously exposed like this is moved to private-libwebsockets.h
so it can be printed along with the rest.
Signed-off-by: Andy Green <andy.green@linaro.org>
- multiple debug context calls lwsl_ err, warn, debug, parser, ext, client
- api added to set which contexts output to stderr using a bitfield log_level
- --disable-debug on configure removes all code that is not err or warn severity
- err and warn contexts always output to stderr unless disabled by log_level
- err and warn enabled by default in log_level
Signed-off-by: Andy Green <andy@warmcat.com>
This patch gets deflate-stream working with x-google-mux.
It adds a clean veto system where are extension can veto the proposal
of any extension when opening a new connection. x-google-mux uses that
in its callback to defeat any use of deflate-stream on mux children.
However deflate stream is allowed on the parent connection and works
transparently now alongside x-google-mux.
Signed-off-by: Andy Green <andy@warmcat.com>
This is initial x-google-mux support. It's disabled by default
since it's very pre-alpha.
1) To enable it, reconfigure with --enable-x-google-mux
2) It conflicts with deflate-stream, use the -u switch on
the test client to disable deflate-stream
3) It deviates from the google standard by sending full
headers in the addchannel subcommand rather than just
changed ones from original connect
4) Quota is not implemented yet
5) Close of subchannel is not really implemented yet
6) Google opcode 0xf is changed to 0x7 to account for
v7 protocol changes to opcode layout
However despite those caveats, in fact it can run the
test client reliably over one socket (both dumb-increment
and lws-mirror-protocol), you can open a browser on the
same test server too and see the circles, etc.
Signed-off-by: Andy Green <andy@warmcat.com>
Mozilla implementationcan issue window of up to 15,
need to match it
Reported-by: Patrick McManus <pmcmanus@mozilla.com>
Signed-off-by: Andy Green <andy@warmcat.com>
Ago noticed that some Windows clients experience small packets
from the server being aggregated and set after a long delay
(200-300ms).
He found that TCP_NODELAY on the socket solved this, I tested it
and it didn't have any noticable bad effect, so I implemented it
for all sockets, client and server.
Thans Ago for debugging this and notifying the cause.
Reported-by: Ago Allikmaa <maxorator@gmail.com>
Signed-off-by: Andy Green <andy@warmcat.com>
