This patch deploys the truncated send work to buffer output in case
either send() or the SSL send return a temporary "unable to send"
condition even though they signalled as writeable.
I added a by-default #if 0 test jig which enforces only half of what
you want to send is sendable, this is working when enabled.
One subtle change is that the pipe reports choked if there is any
pending remaining truncated send. Otherwise it should be transparent.
Hopefully...
Signed-off-by: Andy Green <andy.green@linaro.org>
If the URI coming from the client contains '?' then
- the URI part is terminated with a '\0'
- the remainder of the URI goes in a new header WSI_TOKEN_HTTP_URI_ARGS
- the remainder of the URI is not subject to path sanitization measures (it
still has %xx processing done on it)
In the test server, http requests now also dump header information to stderr.
The attack.sh script is simplified and can now parse the test server header dumps.
Signed-off-by: Andy Green <andy.green@linaro.org>
This translates %xx in the GET uri and removes /.. and /... type sequences along with
translating // or /// etc to /.
Since the result is hopefully secure, it also changes the test server to actually use
the uri path pasted on a resource directory without whitelisting.
Signed-off-by: Andy Green <andy.green@linaro.org>
Clean up minilex
Move the header output to stdout
Introduce lexfile.h as the header output
Use lexfile.h in both minilex itself and lws
Add the following header support
"Accept:",
"If-Modified-Since:",
"Accept-Encoding:",
"Accept-Language:",
"Pragma:",
"Cache-Control:",
"Authorization:",
"Cookie:",
"Content-Type:",
"Date:",
"Range:",
"Referer:"
Signed-off-by: Andy Green <andy.green@linaro.org>
As spotted by JM on Trac#40
http://libwebsockets.org/trac/libwebsockets/ticket/40
client connect didn't do anything about being truly nonblocking. This patch
should hopefully solve that.
Signed-off-by: Andy Green <andy.green@linaro.org>
It is awfully limiting when you have to set the resource path at compile time.
Instead also allow setting it via the --resource_path command line option.
under load, writing packet sizes to the socket that are normally fine
can do partial writes, eg asking to write 4096 may only take 2800 of
it and return 2800 from the actual send.
Until now lws assumed that if it was safe to send, it could take any
size buffer, that's not the case under load.
This patch changes lws_write to return the amount actually taken...
that and the meaning of it becomes tricky when dealing with
compressed links, the amount taken and the amount sent differ. Also
there is no way to recover at the moment from a protocol-encoded
frame only being partially accepted... however for http file send
content it can and does recover now.
Small frames don't have to take any care about it but large atomic
sends (> 2K) have been seen to fail under load.
Signed-off-by: Andy Green <andy.green@linaro.org>
Fixed so that the build options for the CMake project works:
- The test apps used the LWS_NO_EXTENSIONS define, so they needed lws_config.h included when building using CMake.
- Rename some options so that individual test apps can be turned off.
- Separate building the test-client/test-server and compiling the server/client parts into the lib.
- Don't include server or client specific sources into the build if they shouldn't be built.
- Added an error if both client and server parts are excluded at the same time (makes no sense).
- Removed duplicate install targets for the test apps.
- Commented out the WITH_LIBCRYPTO option since it isn't used at the moment.
send() is able to complete but not send everything asked...
we don't deal with that yet. On an ARM9 board, leaf.jpg
failed because the send() only accepted 7240 of 8192... not
a failure but it implies reporting partial send to the
caller. Work around by keeping the send buffer at 4096.
Signed-off-by: Andy Green <andy.green@linaro.org>
The client spams the server with mirror protocol content
and then closes the connection. However the server wants to
send that content back to all the mirror protocol
connections, including the one that closed.
The result is during the send back phase we usually see an
error trying to send to the close client connection. Because
we don't return -1, we don't close it on server side and the
error repeats a lot before finally closing. This is a side-
effect of a recent patch to drain pending input before
really closing a dead connection.
This patch just changes it to return -1 on the failed send
attempt assertively closing the connection.
Signed-off-by: Andy Green <andy.green@linaro.org>
This adds a demonstration of how to use the http nonblocking
send action to the test server.
If you ask for /leaf.jpg from the test server, it will send
"by hand" a 2.4MB jpeg in HTTP, including the headers. See
the test server sources to see how it is done.
Although it's a file, and a jpeg image, actually the exact
same scheme will work for any data or mime type.
Signed-off-by: Andy Green <andy.green@linaro.org>
This now exercises the code for dealing with rx that spills over
the rx buffer. Single fraggle uses the default rx buffer size
of 4096, it also now exercises code around that.
Signed-off-by: Andy Green <andy.green@linaro.org>
- Define LWS_DLL and LWS_INTERNAL when websockets_shared is compiled.
- The websocket_shared target compiles to websocket.lib / websocket.dll
(websocket.lib contains the exported functions for websocket.dll, and is
the file that is linked to when a program wants to use the dll)
- The websocket target compiles to websocket_static.lib on windows.
- Replaced any "extern" with "LWS_EXTERN" on libwebsockets.h for proper
DLL function exports.
- Created a LIB_LIST with all the libwebsocket dependencies, instead of
multiple calls to target_link_libraries, only one call is made for both
the static and shared library version. This makes it easy to add other
variants if wanted in the future.
- Added ZLIB as a dependency for the libs, so that the build order will be
correct at all times.
- Added a dependency for the websockets lib to the test apps, so it is
built before them.
- Fixed the test-server-extpoll app to include the emulated_poll, and link
to winsock on Windows.
- Removed the global export of libwebsocket_internal_extensions, and added
a function libwebsocket_get_internal_extensions() that returns it
instead. Using the global would not work with the DLL export on Windows.
Seems like it would be a good idea to try to mess with the
server at least before someone else does it for us
Just run the script
$ test-server/attack.sh
it will spawn a test server and fire things at it. If you
see the end result
---- survived
then you should be OK.
Signed-off-by: Andy Green <andy.green@linaro.org>
With this, both the test server and client are valgrind-clean
tested with a chrome session also active to the server
Signed-off-by: Andy Green <andy.green@linaro.org>
To get a clean bill of health from valgrind, we have to have a way to
inform the user code that we're going down and it should free everything
it is holding that was malloc'd.
This patch introduces LWS_CALLBACK_PROTOCOL_DESTROY which each protocol
gets when the context is being destroyed and no more activity will come
after that call. They can get rid of everything there.
To match it, LWS_CALLBACK_PROTOCOL_INIT is introduced which would allow
one-time init per protocol too.
Signed-off-by: Andy Green <andy.green@linaro.org>
This big patch replaces the malloc / realloc per header
approach used until now with a single three-level struct
that gets malloc'd during the header union phase and freed
in one go when we transition to a different union phase.
It's more expensive in that we malloc a bit more than 4Kbytes,
but it's a lot cheaper in terms of malloc, frees, heap fragmentation,
no reallocs, nothing to configure. It also moves from arrays of
pointers (8 bytes on x86_64) to unsigned short offsets into the
data array, (2 bytes on all platforms).
The 3-level thing is all in one struct
- array indexed by the header enum, pointing to first "fragment" index
(ie, header type to fragment lookup, or 0 for none)
- array of fragments indexes, enough for 2 x the number of known headers
(fragment array... note that fragments can point to a "next"
fragment if the same header is spread across multiple entries)
- linear char array where the known header payload gets written
(fragments point into null-terminated strings stored in here,
only the known header content is stored)
http headers can legally be split over multiple headers of the same
name which should be concatenated. This scheme does not linearly
conatenate them but uses a linked list in the fragment structs to
link them. There are apis to get the total length and copy out a
linear, concatenated version to a buffer.
Signed-off-by: Andy Green <andy.green@linaro.org>
Noticed previously the test client lifetime is decoupled from
the actual send activity. With SSL if the connection starts but
there is a period of SSL-layer "blocking" (actually fail-and-retry)
the mirror lifetime could be exhausted before the connection really
completed, making it stall after it was then closed.
This corrects that so connection lifetime management is done in the
send callback.
Signed-off-by: Andy Green <andy.green@linaro.org>
*** This patch changes an API all apps use ***
Context creation parameters are getting a bit out of control, this
patch creates a struct to contain them.
All the test apps are updated accordingly.
Signed-off-by: Andy Green <andy.green@linaro.org>
A new protocol member is defined that controls the size of rx
buffer allocation per connection. For compatibility 0 size
allocates 4096, but you should adapt your protocol definition
array in the user code to declare an appropriate value.
See the changelog for more detail.
The advantage is the rx frame buffer size is now tailored to
what is expected from the protocol, rather than being fixed
to a default of 4096. If your protocol only sends frames of
a dozen bytes this allows you to only allocate an rx frame
buffer of the same size.
For example the per-connection allocation (excluding headers)
for the test server fell from ~4500 to < 750 bytes with this.
Signed-off-by: Andy Green <andy.green@linaro.org>
This gets rid of the stack buffer while serving files, and the
PATH_MAX char array that used to hold the filepath in the wsi.
It holds an extra file descriptor open while serving the file,
however it attempts to stuff the socket with as much of the
file as it can take. For files of a few KB, that typically
completes (without blocking) in the call to
libwebsockets_serve_http_file() and then closes the file
descriptor before returning.
Signed-off-by: Andy Green <andy.green@linaro.org>
