diff --git a/doc/openapi/components/schemas/config/nodes/infiniband.yaml b/doc/openapi/components/schemas/config/nodes/infiniband.yaml index 921e240e8..3119ca17a 100644 --- a/doc/openapi/components/schemas/config/nodes/infiniband.yaml +++ b/doc/openapi/components/schemas/config/nodes/infiniband.yaml @@ -4,7 +4,192 @@ allOf: - type: object properties: - format: - $ref: ../format_spec.yaml + + rdma_port_space: + type: string + enum: + - RC + - UC + - UD + default: RC + description: | + This specifies the type of connection the node will set up. + + * `RC` provides reliable, connection-oriented, message based communication between the nodes. Packets are delivered in order. In this mode, one Queue Pair is connected to one other Queue Pair. + * `UC` provides unreliable, connection-oriented, message based communication between the nodes. This service type is not officially supported by the RDMA communication manager and is implemented for scientific purposes in VILLASnode. [The InfiniBand node-type source code provides information on how to enable this service type.](https://git.rwth-aachen.de/acs/public/villas/node/blob/master/lib/nodes/infiniband.c#L429) + * `UD` provides unreliable, connection-less, datagram communication between nodes. Both ordering and delivery are not guaranteed in this mode. + + `RC`, `UC`, and `UD` are mapped to the Queue Pair types as `RDMA_PS_TCP`/`IBV_QPT_RC`, `RDMA_PS_IPOIB`/`IBV_QPT_UC`, and `RDMA_PS_UDP`/`IBV_QPT_UD`, respectively. + If two nodes should be connected, both should be set to the same `rdma_port_space`. + + More information on these two modes can be found on the manual page for [`rdma_create_id()`](https://linux.die.net/man/3/rdma_create_id). + + in: + type: object + properties: + address: + type: string + description: | + Connections between `infiniband` nodes are established over IP over IB (IPoIP). + To use this node, you have to make sure that the linux driver `ib_ipoib` is loaded. + If it is not loaded, load it with `modprobe ib_ipoib`. + + If it is loaded, you have to make sure that the Host Channel Adapters (HCAs) have an IP address. + You can configure the IP address of the Infiniband HCA with the `ifconfig` utility, exactly like you would configure normal Ethernet adapters. + + As soon as an IP is set for the local HCA, this entry can be used to point to the adapter and to define the port which will be used for connection related communication. + + **Example**: + + ``` + in = { + address="10.0.0.1:1337" + } + ``` + + binds the node to the local device which is bound to `10.0.0.1`. It will use port `1337` for communication related to the connection. + + max_wrs: + type: integer + default: 128 + description: | + Before a packet can be received with Infiniband, the application has to describe how this will be handled (e.g., to what address the data will be written). + This happens in a so called Work Request (WR). + + `in.max_wrs` sets the maximum number of receive Work Requests which can be posted to the receive queue of the Queue Pair. + + For higher throughput, it is recommended to increase this value since it will serve as a buffer. + + cq_size: + type: integer + default: 128 + description: | + This value defines the number of Work Completions the Completion Queue can hold. + + If a packet is received, the Queue Pair will write a Work Completion to the Completion Queue. + The node polls this queue to process received packets. If the Completion Queue gets full, which is often caused by `cq_size` being to small, and thus the receive queue is not able to post Work Completions, the node will abort. + + If a connection is disconnected, all outstanding Work Requests—even is they are not used—are flushed to the Completion Queue. + Here applies the same as mentioned above: if the Completion Queue has fewer space left than outstanding Work Requests are available, this will result in an error. + + It is therefor recommended to set the value of `cq_size` to at least + + ``` + in.cq_size >= in.max_wrs - in.buffer_subtraction + ``` + + buffer_subtraction: + type: integer + default: 16 + description: | + As mentioned in the `in.max_wrs` settings, Work Requests have to be present in the receive queue, for it to be able to process received data. + To take full advantage of the zero-copy capabilities of Infiniband this node-type directly posts addresses from the VILLASnode to the receive queue instead of copying all data over after receiving it. + + This technique relies on the exchange of addresses. This means that if an array of `in.vectorize` addresses is handed over to the node-type, max `release` <= `in.vectorize` addresses that point to received data can be returned. + + Furthermore, if `release` addresses should be returned, `release` addresses from the original array must be posted to the receive queue. + To ensure that we can always post at least `in.vectorize` new samples to the receive queue, `in.buffer_subtraction` must always be bigger than `in.vectorize`. + + A second factor is performance: if `in.buffer_subtraction` is too small it might take long before the node starts to process data since it has to fill almost the complete queue first. + If `in.buffer_subtraction` is too big, the receive buffer might be too small. + + Thus, the maximum number of Work Requests to be present in the receive queue is defined as follows: + + ```c + max_wrs_posted = in.max_wrs - in.buffer_subtraction + ``` + out: + type: object + properties: + address: + type: string + description: | + This value defines the IPoIB address of the remote node and is used to establish a connection to the remote host—in case of `RDMA_PS_TCP`—or to get the address handle of the remote host—in case of `RDMA_PS_UDP`. + + This is similar to `in.address`. + + `out.address` has no default value and if it is not defined the node will be set to listening mode and all `out` configuration will be ignored. + + **Example**: + + ``` + out = { + address = "10.0.0.1:1337" + } + ``` + + timeout: + type: integer + default: 1000 + description: | + This defines the time in milliseconds [`rdma_resolve_addr()`](https://linux.die.net/man/3/rdma_resolve_addr) waits for the resolution of the destination address to complete. + + max_wrs: + type: integer + default: 128 + description: | + This is similar to `in.max_wrs` but for the send side of the Queue Pair. + In contrast to the receive queue, there is no minimum amount of Work Requests in this queue and it can be filled up completely to `out.max_wrs`. + + cq_size: + type: integer + default: 128 + description: | + This is similar to `in.cq_size`. + + An important side note for the receive completion queue was that it should be able to hold all Work Requests if the receive queue is flushed. + Since no "preparatory" Work Requests are posted to the send queue and and thus all work requests are send out as soon as possible, there is no need for `out.cq_size` to be as big as `out.max_wrs`. + + send_inline: + type: boolean + default: true + description: | + It is possible that the CPU copies the data to be sent directly to the HCA. + Then, the HCA can take the data from it's internal memory as soon as it is ready to send it. + This has the advantage that the buffer can be returned immediately to the VILLASnode and that it increases performance. + + If this flag is set, the [`infiniband`](../nodes/infiniband.md) node-type checks if a sample is small enough to be sent inline, and if this is the case sends it inline. + + max_inline_data: + type: integer + default: 0 + description: | + This value represents the maximum number of bytes to be send inline. + The maximum number of this value depends on the HCA. + The settings defaults to zero. However, many HCAs will automatically adjust it to 60. + + *Important note*: The greater this value gets, the smaller `out.max_wrs` can be. If `out.max_inline_data` is too big for the number specified in `out.max_wrs`, the node will return an error that the Queue Pair could not be created. + Since this is different for various HCAs, it is not possible for us to give more specified errors. + + **Example**: + + ``` + out = { + send_inline = 1, + max_inline_data = 60 + } + ``` + + Every sample which is smaller than 60 bytes will be send inline. All other samples will be sent normally. + + use_fallback: + type: boolean + default: true + description: | + If an out section with a valid remote entry is present in the configuration file, the node will first bind to the local host channel adapter and subsequentially try to connect to the remote host. + If the latter fails (e.g., because the remote host was not reachable or rejected the connection), there are two possible outcomes: the node can throw an error and abort or it can show a warning and continue in listening mode. + + If `use_fallback = true`, the node will fallback to listening mode if it is not able to connect to the remote host. + + periodic_signaling: + type: integer + default: + description: | + If a sample is sent inline, no Completion Queue Entry (CQE) is generated. + However, once a while, a CQE must be generated to prevent the Send Queue from overflowing. + Therefore, every `out.periodic_signaling`th sample will be sent normally with signaling. + + It turns out that the ideal value in most cases is `out.max_wrs / 2`. + Hence, usually, it is not necessary to explicitly set this value. - $ref: ../node.yaml diff --git a/doc/openapi/components/schemas/config/nodes/influxdb.yaml b/doc/openapi/components/schemas/config/nodes/influxdb.yaml index 921e240e8..69387e327 100644 --- a/doc/openapi/components/schemas/config/nodes/influxdb.yaml +++ b/doc/openapi/components/schemas/config/nodes/influxdb.yaml @@ -4,7 +4,15 @@ allOf: - type: object properties: - format: - $ref: ../format_spec.yaml + server: + type: string + description: A hostname/port combination of the InfluxDB database server. + + key: + type: string + description: | + The key is the measurement name and any optional tags separated by commas. + + See also: [InfluxDB documentation](https://docs.influxdata.com/influxdb/v0.9/write_protocols/line/#key). - $ref: ../node.yaml diff --git a/doc/openapi/components/schemas/config/nodes/kafka.yaml b/doc/openapi/components/schemas/config/nodes/kafka.yaml index 921e240e8..6e1663ffa 100644 --- a/doc/openapi/components/schemas/config/nodes/kafka.yaml +++ b/doc/openapi/components/schemas/config/nodes/kafka.yaml @@ -3,8 +3,80 @@ allOf: - type: object + required: + - server + - client_id properties: format: $ref: ../format_spec.yaml + server: + type: string + description: | + The bootstrap server `{ip}:{port}` of the Kafka message brokers cluster. + + protocol: + type: string + enum: + - PLAINTEXT + - SASL_PLAINTEXT + - SASL_SSL + - SSL + description: | + The [security protocol](https://kafka.apache.org/24/javadoc/org/apache/kafka/common/security/auth/SecurityProtocol.html) which is used for authentication with the Kafka cluster. + + client_id: + type: string + description: The Kafka client identifier. + + ssl: + type: object + properties: + ca: + type: string + description: Path to a Certificate Authority (CA) bundle which is used to validate broker server certificate. + + sasl: + type: object + description: | + An object for configuring the SASL authentication against the broker. + This setting is used if the `protocol` setting is on of `SASL_PLAINTEXT` or `SASL_SSL`. + + properties: + mechanisms: + type: string + + username: + type: string + + password: + type: string + + in: + type: object + properties: + consume: + type: string + description: The Kafka topic to which this node-type will subscribe for receiving messages. + + group_id: + type: string + description: The group id of the Kafka client used for receiving messages. + + out: + type: object + properties: + produce: + type: string + description: The Kafka topic to which this node-type will publish messages. + + timeout: + type: number + description: A timeout in seconds for the broker connection. + default: 1.0 + + + + + - $ref: ../node.yaml diff --git a/doc/openapi/components/schemas/config/nodes/ngsi.yaml b/doc/openapi/components/schemas/config/nodes/ngsi.yaml index 921e240e8..0ddb2e0b2 100644 --- a/doc/openapi/components/schemas/config/nodes/ngsi.yaml +++ b/doc/openapi/components/schemas/config/nodes/ngsi.yaml @@ -3,8 +3,24 @@ allOf: - type: object + required: + - endpoint properties: - format: - $ref: ../format_spec.yaml + endpoint: + type: string + format: uri + + entity_id: + type: string + + entity_type: + type: string + + ssl_verify: + type: boolean + + timeout: + type: number + - $ref: ../node.yaml diff --git a/doc/openapi/components/schemas/config/nodes/redis.yaml b/doc/openapi/components/schemas/config/nodes/redis.yaml index 921e240e8..714298e3a 100644 --- a/doc/openapi/components/schemas/config/nodes/redis.yaml +++ b/doc/openapi/components/schemas/config/nodes/redis.yaml @@ -4,7 +4,130 @@ allOf: - type: object properties: - format: - $ref: ../format_spec.yaml + mode: + type: string + enum: + - key + - hash + - channel + default: key + description: | + - `key`: [Get](https://redis.io/commands/get)/[Set](https://redis.io/commands/set) of [Redis strings](https://redis.io/topics/data-types#strings) + - The implementation uses the Redis `MSET` and `MGET` commands. + - `hash`: Hashtables using [hash data-type](https://redis.io/topics/data-types#hashes) + - The implementation uses the Redis `HMSET` and `HGETALL` commands. + - `channel`: [Publish/subscribe](https://redis.io/topics/pubsub) + - The implementation uses the Redis `PUBLISH` and `SUBSCRIBE` commands. + + uri: + type: string + format: uri + description: | + A Redis connection URI in the form of: `redis://:@:/`. + + host: + type: string + default: localhost + description: | + The hostname or IP address of the Redis server. + + You can also connect to Redis server with a URI: + + - `tcp://[[username:]password@]host[:port][/db]` + - `unix://[[username:]password@]path-to-unix-domain-socket[/db]` + + port: + type: integer + description: The port number of the Redis server to connect to. + default: 6379 + + path: + type: string + description: A path of a Unix socket which should be used for the connection. + + user: + type: string + default: default + description: | + The username which should be used for authentication. + + See: https://redis.io/commands/auth + + password: + type: string + description: | + The password which should be used for authentication. + + See: https://redis.io/commands/auth + + + db: + type: integer + default: 0 + description: | + The logical database which should be used by the Redis client. + + See: https://redis.io/commands/select + + timeout: + type: object + properties: + connect: + type: number + description: The timeout in seconds for the initial connection establishment. + + socket: + type: number + description: The timeout in seconds for executing commands against the Redis server. + + keepalive: + type: boolean + default: false + description: Enable periodic keepalive packets. + + key: + type: string + default: + description: The key which this node will use in the Redis keyspace. + + channel: + type: string + default: + description: The channel which this node will use when `mode` setting is `channel`. + + notify: + type: boolean + default: true + description: | + Use [Redis keyspace notifications](https://redis.io/topics/notifications) to listen for new updates. + This setting is only used if setting `mode` is set to `key` or `hash`. + + ssl: + type: object + properties: + enabled: + type: boolean + default: true + description: If enabled the connection to the Redis server will be encrypted via SSL/TLS. + + cacert: + type: string + description: A path to a CA certificate file. + + cacertdir: + type: string + description: A path to a directory containing CA certificates. + + cert: + type: string + description: A path to a client certificate file. + + key: + type: string + description: A path to the private key file. + + + + - $ref: ../node.yaml diff --git a/doc/openapi/components/schemas/config/nodes/rtp.yaml b/doc/openapi/components/schemas/config/nodes/rtp.yaml index 921e240e8..26a45ead0 100644 --- a/doc/openapi/components/schemas/config/nodes/rtp.yaml +++ b/doc/openapi/components/schemas/config/nodes/rtp.yaml @@ -7,4 +7,10 @@ allOf: format: $ref: ../format_spec.yaml + out: + type: object + properties: + netem: + $ref: ../netem.yaml + - $ref: ../node.yaml diff --git a/doc/openapi/components/schemas/config/nodes/shmem.yaml b/doc/openapi/components/schemas/config/nodes/shmem.yaml index 921e240e8..91d403acd 100644 --- a/doc/openapi/components/schemas/config/nodes/shmem.yaml +++ b/doc/openapi/components/schemas/config/nodes/shmem.yaml @@ -4,7 +4,52 @@ allOf: - type: object properties: - format: - $ref: ../format_spec.yaml + queuelen: + type: integer + default: + description: Length of the input and output queues in elements. + + samplelen: + type: integer + description: Maximum number of data elements in a single `struct Sample`` for the samples handled by this node. + default: + + mode: + type: string + default: pthread + enum: + - pthread + - polling + description: | + If set to `pthread`, POSIX condition variables (CV) are used to signal writes between processes. + If set to `polling`, no CV's are used, meaning that blocking writes have to be implemented using polling, leading to performance improvements at a cost of unnecessary CPU usage. + + exec: + description: | + Optional name and command-line arguments (as passed to `execve`) of a command to be executed during node startup. + This can be used to start the external program directly from VILLASNode. If unset, no command is executed. + type: array + items: + type: string + + in: + type: object + properties: + name: + type: string + description: | + Name of the POSIX shared memory object. + Must start with a forward slash (/). + The same name should be passed to the external program somehow in its configuration or command-line arguments. + + out: + type: object + properties: + name: + type: string + description: | + Name of the POSIX shared memory object. + Must start with a forward slash (/). + The same name should be passed to the external program somehow in its configuration or command-line arguments. - $ref: ../node.yaml diff --git a/doc/openapi/components/schemas/config/nodes/signal.yaml b/doc/openapi/components/schemas/config/nodes/signal.yaml index 921e240e8..67444205d 100644 --- a/doc/openapi/components/schemas/config/nodes/signal.yaml +++ b/doc/openapi/components/schemas/config/nodes/signal.yaml @@ -3,8 +3,103 @@ allOf: - type: object + required: + - signal properties: - format: - $ref: ../format_spec.yaml + signal: + type: string + enum: + - random + - sine + - square + - triangle + - ramp + - counter + - constant + - mixed + - pulse + description: | + The type of signal which should be generated: + + - `random`: a random walk with normal distributed step sizes will be generated. + - `sine`: a sine signal will be generated. + - `square`: a square / rectangle wave will be generated. + - `triangle`: a triangle wave will be generated. + - `ramp`: the generator will produce a ramp signal in the interval `[ 0, 1 / f ]`. + - `counter`: increasing integer counter is generated. + - `constant`: a constant value generated. + - `mixed`: the signals of of each sample are generated by cycling over all remaining signal types. + - `pulse`: generates pulses with a set frequency, phase and width + + values: + type: integer + default: 1 + description: The number of signals which each of the generated samples should contain. + + rate: + type: integer + description: The rate at which sample should be generated by the node. + default: 10 + + amplitude: + type: number + description: The amplitude of the signal when the `signal` setting is one of `sine`, `square` or `triangle`. + default: 1.0 + + frequency: + type: number + description: The frequency of the signal when the `signal` setting is one of `sine`, `square`, `triangle`,`pulse` or `ramp`. + default: 1.0 + + phase: + type: number + default: 0.0 + description: Tha pase of the signal when the `signal` setting is one of `sine` or `pulse`. + + + pulse_width: + type: number + default: 1.0 + description: The width of the pulse, with respect to the rate + + pulse_low: + type: number + default: 0.0 + description: The low value of the pulse signal. + + pulse_high: + type: number + default: 1.0 + description: The high value of the pulse signal. + + stddev: + type: number + default: 0.2 + description: The standard deviation of the normal distributed steps if the `signal` setting is set to `random`. + + offset: + type: number + default: 0.0 + description: Adds a constant offset to each of the generated signals. + + limit: + type: integer + default: -1 + description: | + Limit the number of generated output samples by this node-type. + A negative number disables the limitation. + + realtime: + type: boolean + default: true + description: Wait `1 / rate` seconds between emitting each sample. + + monitor_missed: + type: boolean + default: true + description: | + If `true`, the `signal` node-type will count missed steps and warn the user during every iteration about missed steps. + Especially at high rates, it can be beneficial for performance to set this flag to `false`. + Warnings would namely cause system calls which will slow the node down even more, and thus cause even more missed steps. - $ref: ../node.yaml diff --git a/doc/openapi/components/schemas/config/nodes/temper.yaml b/doc/openapi/components/schemas/config/nodes/temper.yaml index 921e240e8..c8e2a4adb 100644 --- a/doc/openapi/components/schemas/config/nodes/temper.yaml +++ b/doc/openapi/components/schemas/config/nodes/temper.yaml @@ -4,7 +4,26 @@ allOf: - type: object properties: - format: - $ref: ../format_spec.yaml + calibration: + type: object + properties: + scale: + type: number + default: 1.0 + description: A scaling factor for calibrating the sensor. + + offset: + type: number + default: 0.0 + description: An offset for calibrating the sensor. + + bus: + type: integer + description: A filter applied to the USB bus number for selecting a specific sensor if multiple are available. + + port: + type: integer + description: A filter applied to the USB port number for selecting a specific sensor if multiple are available. + - $ref: ../node.yaml