mirror of
https://git.rwth-aachen.de/acs/public/villas/node/
synced 2025-03-09 00:00:00 +01:00
updating documentation
This commit is contained in:
parent
41305049e2
commit
9a7e95691d
12 changed files with 200 additions and 160 deletions
|
|
@ -2,6 +2,8 @@
|
|||
|
||||
@subpage gtfpga
|
||||
@subpage opal
|
||||
@subpage file
|
||||
@subpage socket
|
||||
|
||||
Every server needs clients which act as sinks / sources for simulation data. In case of S2SS these clients are called _nodes_.
|
||||
|
||||
|
|
|
|||
|
|
@ -21,53 +21,26 @@ Possible types of nodes are:
|
|||
|
||||
## Paths
|
||||
|
||||
A path is a **unidirectional** connection between incoming and outgoing nodes.
|
||||
A path is a **uni-directional** connection between incoming and outgoing nodes.
|
||||
|
||||
A path simply forwards messages from the incoming node to the outgoing node.
|
||||
By default the server does not alter message contents.
|
||||
It only checks if the message headers are valid (sequence number, cryptographic signature..)
|
||||
However every path supports optional callback function which allow user-defined operations on the message contents.
|
||||
It forwards messages from a single incoming node to multiple outgoing nodes.
|
||||
Therefore it represents a 1-to-n relation between nodes.
|
||||
|
||||
For bidirectional communication a corresponding path in the reverse direction must be added.
|
||||
|
||||
The server is designed to allow multiple outgoing and incoming nodes per path as an extension.
|
||||
@todo This has **not** been implemented at the moment but will come in future versions.
|
||||
|
||||
By default, message contents are not altered.
|
||||
The server only performs checks for valid message headers (sequence number, cryptographic signature..).
|
||||
However every path supports optional hook/callback functions which allow user-defined operations on the message contents.
|
||||
|
||||
@diafile path_simple.dia
|
||||
|
||||
@see path for implementation details.
|
||||
|
||||
## Message
|
||||
|
||||
A message contains a variable number of values.
|
||||
Usually a a simulator sends one message per timestep.
|
||||
|
||||
Simulation data is encapsulated in UDP packages in sent over IP networks like the internet.
|
||||
We designed a lightweight message format (or protocol) to facilitate a fast transmission.
|
||||
|
||||
@diafile msg_format.dia
|
||||
|
||||
### Protocol details
|
||||
|
||||
There are several types of messages:
|
||||
|
||||
* **Data**: contains simulation data.
|
||||
* **Control**: start/stops a new simulation case.
|
||||
* **Config**: used to pass settings between nodes.
|
||||
* **Sync**: used to syncrhonize the clocks between nodes.
|
||||
|
||||
For now, only the first type (data) is implemented.
|
||||
Therefore the complete protocol is **stateless**.
|
||||
Later we might want to support more complex simulation scenarios which require some kind of controlling.
|
||||
|
||||
Except for the simulation data, all values are sent in **network byte order** (big endian)!
|
||||
The endianess of the simulation data is indicated by a single bit in the message header.
|
||||
This allows us to reduce the amount of conversions during one transfer.
|
||||
|
||||
@see msg for implementation details.
|
||||
|
||||
## Interface
|
||||
|
||||
Interfaces are used to represent physical network ports on the server.
|
||||
They are only used by the [Socket](socket) type of nodes.
|
||||
|
||||
@todo Add documentation
|
||||
|
||||
@see interface for implementation details.
|
||||
|
|
|
|||
|
|
@ -1,18 +1,22 @@
|
|||
# Configuration
|
||||
|
||||
The S2SS server configuration is completly done in a single file.
|
||||
The S2SS configuration is completly contained in a single file.
|
||||
Take a look at the example configuration: `server/etc/example.conf`.
|
||||
|
||||
The configuration file mainly consists of three sections:
|
||||
The configuration file consists of three sections:
|
||||
|
||||
### Global
|
||||
## Global
|
||||
|
||||
The global section consists of some global configuration parameters:
|
||||
|
||||
#### `debug`
|
||||
|
||||
`debug` expects an integer number (0-10) which determines the verbosity of debug messages during the execution of the server.
|
||||
Use this with care! Producing a lot of IO might decrease the performance of the server.
|
||||
Omitting this setting or setting it to zero will disable debug messages completely.
|
||||
|
||||
#### `stats`
|
||||
|
||||
`stats` specifies the rate in which statistics about the actives paths will be printed to the screen.
|
||||
Setting this value to 5, will print 5 lines per second.
|
||||
A line of includes information such as:
|
||||
|
|
@ -21,38 +25,44 @@ A line of includes information such as:
|
|||
- Messages sent
|
||||
- Messaged dropped
|
||||
|
||||
#### `affinity`
|
||||
|
||||
The `affinity` setting allows to restrict the exeuction of the daemon to certain CPU cores.
|
||||
This technique, also called 'pinning', improves the determinism of the server by isolating the daemon processes on exclusive cores.
|
||||
|
||||
#### `priority`
|
||||
|
||||
The `priority` setting allows to adjust the scheduling priority of the deamon processes.
|
||||
By default, the daemon uses a real-time optimized FIFO scheduling algorithm.
|
||||
|
||||
### Nodes
|
||||
## Nodes
|
||||
|
||||
The node section is a directory of nodes (clients) which are connected to the S2SS instance.
|
||||
The node section is a **directory** of nodes (clients) which are connected to the S2SS instance.
|
||||
The directory is indexed by the name of the node:
|
||||
|
||||
nodes = {
|
||||
"sintef_node" = {
|
||||
type = "udp"
|
||||
type = "socket"
|
||||
....
|
||||
}
|
||||
}
|
||||
|
||||
There are multiple diffrent type of nodes. But all types have the following settings in common:
|
||||
|
||||
#### `type`
|
||||
|
||||
`type` sets the type of the node. This should be one of:
|
||||
- `udp`
|
||||
- `ip`
|
||||
- `gtfpga`
|
||||
- `opal`
|
||||
- `file`
|
||||
- `socket` which refers to a [Socket](socket) node.
|
||||
- `gtfpga` which refers to a [GTFPGA](gtfpga) node.
|
||||
- `opal` which refers to a [OPAL Asynchronous Process](opal) node.
|
||||
- `file` which refers to a [File](file) node.
|
||||
|
||||
For a detailed description of the node-specific settings, please
|
||||
The remaining settings per node a depending on `type`.
|
||||
Take a look a the specific pages for details.
|
||||
|
||||
### Paths
|
||||
## Paths
|
||||
|
||||
The path section consists of a list of paths.
|
||||
The path section consists of a **list** of paths.
|
||||
Every path is allowed to have the following settings:
|
||||
|
||||
The `in` and `out` settings expect the name of the source and destination node.
|
||||
|
|
|
|||
|
|
@ -1,8 +1,4 @@
|
|||
# Server
|
||||
|
||||
@subpage configuration
|
||||
@subpage usage
|
||||
@subpage netem
|
||||
# Setup
|
||||
|
||||
## Compilation
|
||||
|
||||
|
|
@ -10,12 +6,16 @@
|
|||
|
||||
Install libraries including developement headers for:
|
||||
|
||||
- _libconfig_ for parsing the config file
|
||||
- [libconfig](http://www.hyperrealm.com/libconfig/) for parsing the config file
|
||||
- [libnl3](http://www.infradead.org/~tgr/libnl/) for the network emulation support
|
||||
|
||||
Use the following command to install the dependencies under Debian-based distributions:
|
||||
|
||||
$ sudo apt-get install iproute2 libconfig-dev libsodium-dev
|
||||
or:
|
||||
$ sudo apt-get install iproute2 libconfig-dev linbl-3-dev
|
||||
|
||||
$ sudo yum install iproute2 libconfig-devel libsodium-devel
|
||||
or the following line for Fedora / CentOS / Redhat systems:
|
||||
|
||||
$ sudo yum install iproute2 libconfig-devel libnl3-devel
|
||||
|
||||
**Important:** Please note that the server requires the
|
||||
[iproute2](http://www.linuxfoundation.org/collaborate/workgroups/networking/iproute2)
|
||||
|
|
|
|||
|
|
@ -1,2 +1,3 @@
|
|||
# Timing
|
||||
|
||||
@diafile timing.dia
|
||||
|
|
|
|||
29
documentation/Tools.md
Normal file
29
documentation/Tools.md
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
# Tools {#tools}
|
||||
|
||||
S2SS comes with a couple of tools to test and debug:
|
||||
|
||||
1. `send` can be used to send samples from STDIN to a node
|
||||
2. `receive` can be used to read samples from a node and write it to STDOUT
|
||||
3. `random` writes random sample data to STDOUT
|
||||
|
||||
## Examples
|
||||
|
||||
1. Start server:
|
||||
|
||||
$ ./server etc/example.conf
|
||||
|
||||
2. Receive/dump data to file
|
||||
|
||||
$ ./receive etc/example.conf node_name > dump.csv
|
||||
|
||||
3. Replay recorded data:
|
||||
|
||||
$ ./send etc/example.conf -r node_name < dump.csv
|
||||
|
||||
4. Send random generated values:
|
||||
|
||||
$ ./random 4 100 | sudo ./send etc/example.conf destination_node
|
||||
|
||||
5. Test ping/pong latency:
|
||||
|
||||
$ ./test latency etc/example.conf test_node
|
||||
|
|
@ -1,24 +1,6 @@
|
|||
# Configuration {#configuration}
|
||||
# Tuning {#tuning}
|
||||
|
||||
## File
|
||||
|
||||
The server gets exclusively configured by its configuration file.
|
||||
|
||||
The following sections describes the available options.
|
||||
Take a look in the 'Examples' section for some examples.
|
||||
|
||||
### Global
|
||||
|
||||
- *name*:
|
||||
- *affinity*
|
||||
|
||||
### Nodes
|
||||
|
||||
### Paths
|
||||
|
||||
## Tuning
|
||||
|
||||
### Operating System and Kernel
|
||||
## Operating System and Kernel
|
||||
|
||||
For minimum latency several kernel and driver settings can be optimized.
|
||||
A [RT-preemp patched Linux](https://rt.wiki.kernel.org/index.php/Main_Page) kernel is recommended.
|
||||
|
|
@ -39,7 +21,7 @@ Precompiled kernels for Fedora can be found here: http://ccrma.stanford.edu/plan
|
|||
# https://www.kernel.org/doc/Documentation/networking/e1000e.txt
|
||||
option e1000e InterruptThrottleRate=
|
||||
|
||||
### Hardware
|
||||
## Hardware
|
||||
|
||||
This are some proposals for the selection of appropriate server hardware:
|
||||
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
# Usage Instructions {#usage}
|
||||
# Usage {#usage}
|
||||
|
||||
The S2SS server (`server`) expects the path to a configuration file as a single argument.
|
||||
|
||||
|
|
@ -35,11 +35,11 @@ The server requires root privileges for:
|
|||
|
||||
3. Go to S2SS directory
|
||||
|
||||
$ cd /home/acs-admin/msv/s2ss/server/
|
||||
$ cd /home/acs-admin/msv/s2ss/server/
|
||||
|
||||
4. Edit configuration file
|
||||
|
||||
$ nano etc/opal-test.conf
|
||||
$ nano etc/opal-test.conf
|
||||
|
||||
- Take a look at the examples and documentation for a detailed description
|
||||
- Move with: cursor keys
|
||||
|
|
@ -47,7 +47,7 @@ The server requires root privileges for:
|
|||
|
||||
5. Start server
|
||||
|
||||
$ sudo ./server etc/opal-test.conf
|
||||
$ sudo ./server etc/opal-test.conf
|
||||
|
||||
- `sudo` starts the server with super user priviledges
|
||||
|
||||
|
|
@ -55,26 +55,4 @@ The server requires root privileges for:
|
|||
|
||||
7. Logout
|
||||
|
||||
$ exit
|
||||
|
||||
## Examples
|
||||
|
||||
1. Start server:
|
||||
|
||||
$ ./server etc/example.conf
|
||||
|
||||
2. Receive/dump data to file
|
||||
|
||||
$ ./receive *:10200 > dump.csv
|
||||
|
||||
3. Replay recorded data:
|
||||
|
||||
$ ./send 4 192.168.1.12:10200 < dump.csv
|
||||
|
||||
4. Send random generated values:
|
||||
|
||||
$ ./random 1 4 100 | ./send 4 192.168.1.12:10200
|
||||
|
||||
5. Test ping/pong latency:
|
||||
|
||||
$ ./test latency 192.168.1.12:10200
|
||||
$ exit
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
# File
|
||||
# File {#file}
|
||||
|
||||
The `file` node-type can be used to log or replay sample values to disk.
|
||||
|
||||
|
|
@ -6,13 +6,43 @@ The `file` node-type can be used to log or replay sample values to disk.
|
|||
|
||||
Every `file` node supports the following settings:
|
||||
|
||||
`in` specifies the path to a file which contains data for replaying.
|
||||
#### `in`
|
||||
|
||||
`out` specifies the path to a file where samples will be dumped.
|
||||
Specifies the path to a file which contains data for replaying.
|
||||
See below for a description of the file format.
|
||||
|
||||
`mode`
|
||||
#### `out`
|
||||
|
||||
`rate`
|
||||
Specifies the path to a file where samples will be written to.
|
||||
This setting allows to add special paceholders for time and date values.
|
||||
See [strftime(3)](http://man7.org/linux/man-pages/man3/strftime.3.html) for a list of supported placeholder.
|
||||
|
||||
**Example**:
|
||||
|
||||
out = "logs/measurements_%Y-%m-%d_%H-%M-%S.log"
|
||||
|
||||
will create a file called: *path_of_working_directory*/logs/measurements_2015-08-09_22-20-50.log
|
||||
|
||||
#### `file_mode`
|
||||
|
||||
Specifies the mode which should be used to open the output file.
|
||||
See [open(2)](http://man7.org/linux/man-pages/man2/open.2.html) for an explanation of allowed values.
|
||||
The default value is `w+` which will start writing at the beginning of the file and create it in case it does not exist yet.
|
||||
|
||||
#### `epoch_mode`
|
||||
|
||||
This setting allows to select the behaviour of the following `epoch` setting.
|
||||
It can be used to adjust the point in time when the first value should be read.
|
||||
|
||||
The behaviour of `epoch` is depending on the value of `epoch_mode`.
|
||||
|
||||
- `epoch_mode = now`: The first value is read at *now* + `epoch` seconds.
|
||||
- `epoch_mode = relative`: The first value is read at *start* + `epoch` seconds.
|
||||
- `epoch_mode = absolute`: The first value is read at `epoch` seconds after 1970-01-01 00:00:00.
|
||||
|
||||
#### `rate`
|
||||
|
||||
By default `rate` has the value `0`. If the value is non-zero,
|
||||
|
||||
### Example
|
||||
|
||||
|
|
@ -26,9 +56,18 @@ Every line in a file correspondents to a message / sample of simulation data.
|
|||
The columns of a line are seperated by whitespaces (tabs or spaces).
|
||||
The columns are defined as follows:
|
||||
|
||||
1. Floating point timestamp in seconds since 1970-01-01 00:00:00 (aka Unix Epoch ).
|
||||
2. Up to `MSG_VALUES` floating point values per sample. The values are seperated by whitespaces as well.
|
||||
1. Seconds in floating point format since 1970-01-01 00:00:00 (aka Unix epoch timestamp: `date +%s`).
|
||||
2. Sequence number
|
||||
3. Up to `MSG_VALUES` floating point values per sample. The values are seperated by whitespaces as well.
|
||||
|
||||
### Example
|
||||
|
||||
@todo Add example dump
|
||||
This example shows a dump with three values per sample:
|
||||
|
||||
1438959964.162102394 6 3.489760 -1.882725 0.860070
|
||||
1438959964.261677582 7 2.375948 -2.204084 0.907518
|
||||
1438959964.361622787 8 3.620115 -1.359236 -0.622333
|
||||
1438959964.461907066 9 5.844254 -0.966527 -0.628751
|
||||
1438959964.561499526 10 6.317059 -1.716363 0.351925
|
||||
1438959964.661578339 11 6.471288 -0.159862 0.123948
|
||||
1438959964.761956859 12 7.365932 -1.488268 -0.780568
|
||||
|
|
|
|||
|
|
@ -10,16 +10,13 @@ The PowerPC core is used to forward values between RTDS and the S2SS server.
|
|||
|
||||
## Configuration
|
||||
|
||||
`slot`
|
||||
#### `slot`
|
||||
|
||||
`id`
|
||||
#### `id`
|
||||
|
||||
`rate`
|
||||
#### `rate`
|
||||
|
||||
## Source Code
|
||||
### Example
|
||||
|
||||
The source code for the PowerPC is located at:
|
||||
@todo Add excerpt from example.conf
|
||||
|
||||
/clients/gtfpga/s2ss/
|
||||
|
||||
@todo Add documentation for GTFPGA card.
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
# Socket
|
||||
# Socket {#socket}
|
||||
|
||||
The socket node-type is the most comprehensive and complex one.
|
||||
It allows to send and receive simulation data over the network.
|
||||
|
|
@ -15,13 +15,13 @@ The implementation supports multiple protocols / OSI layers:
|
|||
|
||||
## Configuration
|
||||
|
||||
`local`
|
||||
#### `local`
|
||||
|
||||
`remote`
|
||||
#### `remote`
|
||||
|
||||
`netem`
|
||||
#### `netem`
|
||||
|
||||
`layer`
|
||||
#### `layer`
|
||||
|
||||
### Example
|
||||
|
||||
|
|
@ -29,10 +29,68 @@ The implementation supports multiple protocols / OSI layers:
|
|||
|
||||
## Packet Format
|
||||
|
||||
@todo add DIA figure here
|
||||
The on-wire format of the network packets is not subject to a standardization process.
|
||||
It's a very simple packet-based format which includes:
|
||||
|
||||
- 32bit floating-point or integer values
|
||||
- 32bit timestamp (integral seconds)
|
||||
- 32bit timestamp (integral nanoseconds)
|
||||
- 16bit sequence number
|
||||
- 4bit version identified
|
||||
- and several flags...
|
||||
|
||||
## Message
|
||||
|
||||
A message contains a variable number of values.
|
||||
Usually a a simulator sends one message per timestep.
|
||||
|
||||
Simulation data is encapsulated in UDP packages in sent over IP networks like the internet.
|
||||
We designed a lightweight message format (or protocol) to facilitate a fast transmission.
|
||||
|
||||
@diafile msg_format.dia
|
||||
|
||||
For now, only the first message type (`data`) is used.
|
||||
Therefore the complete protocol is **stateless**.
|
||||
Later we might want to support more complex simulation scenarios which require some kind of controlling.
|
||||
|
||||
Except for the simulation data, all values are sent in **network byte order** (big endian)!
|
||||
The endianess of the simulation data is indicated by a single bit in the message header.
|
||||
This allows us to reduce the amount of conversions during one transfer.
|
||||
|
||||
@see msg for implementation details.
|
||||
|
||||
### Example
|
||||
|
||||
@todo add screenshot of wireshark dump
|
||||
|
||||
## Network Emulation
|
||||
## Network Emulation {#netem}
|
||||
|
||||
S2SS supports the emulation of wide-area network characterisics.
|
||||
|
||||
This emulation can be configured on a per-node basis for **outgoing** / **egress** data only.
|
||||
Incoming data is not processed by the network emulation!
|
||||
|
||||
This network emulation is handled by Linux' [netem queuing discipline](http://www.linuxfoundation.org/collaborate/workgroups/networking/netem) which is part of the traffic control subsystem.
|
||||
Take a look at the following manual page for supported metrics: [tc-netem(8)](http://man7.org/linux/man-pages/man8/tc-netem.8.html).
|
||||
|
||||
S2SS only takes care of setup and initalizing the netem queuing discipline inside the kernel.
|
||||
For this the iproute2 software package (`ip` & `tc` commands) must be installed.
|
||||
The configuration is done via the config file.
|
||||
Look at `etc/example.conf` for a section called `netem` or `tc-netem(8)` for more details.
|
||||
|
||||
### Custom delay distribution
|
||||
|
||||
Netem supports loading custom delay distributions.
|
||||
|
||||
1. Load and compile the netem tools from:
|
||||
https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/netem
|
||||
2. Create a custom distribution by following the steps described here:
|
||||
https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/README.distribution
|
||||
3. Put the generated distrubtion with the suffix `.dist` in the `tc` lib directory: `/usr/lib/tc/`.
|
||||
4. Load the distribution specifying the basename in the server config.
|
||||
|
||||
### Further information
|
||||
|
||||
- https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/README.iproute2+tc
|
||||
- https://github.com/stv0g/netem
|
||||
|
||||
|
|
|
|||
|
|
@ -1,29 +0,0 @@
|
|||
# Network Emulation {#netem}
|
||||
|
||||
S2SS supports the emulation of wide-area network characterisics.
|
||||
|
||||
For this an existing network emulator called 'netem' is used.
|
||||
Netem is part of the Linux kernel and therefore shipped with most distributions by default.
|
||||
Netem is implemented as a queuing discipline.
|
||||
This discipline define the order and point in time when packets will be send over the network.
|
||||
|
||||
S2SS only takes care of setup and initalizing the netem queuing discipline inside the kernel.
|
||||
For this the iproute2 software package (`ip` & `tc` commands) must be installed.
|
||||
The configuration is done via the config file.
|
||||
Look at `etc/example.conf` for a section called `netem` or `tc-netem(8)` for more details.
|
||||
|
||||
## Custom delay distributions
|
||||
|
||||
Netem supports loading custom delay distributions.
|
||||
|
||||
1. Load and compile the netem tools from:
|
||||
https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/netem
|
||||
2. Create a custom distribution by following the steps described here:
|
||||
https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/README.distribution
|
||||
3. Put the generated distrubtion with the suffix `.dist` in the `tc` lib directory: `/usr/lib/tc/`.
|
||||
4. Load the distribution specifying the basename in the server config.
|
||||
|
||||
## Links
|
||||
|
||||
- http://www.linuxfoundation.org/collaborate/workgroups/networking/netem
|
||||
- https://git.kernel.org/cgit/linux/kernel/git/shemminger/iproute2.git/tree/README.iproute2+tc
|
||||
Loading…
Add table
Reference in a new issue