Performance Tuning
Warning
This page is under construction and currently only providing very limited information.
Tuning nzyme-node
The performance section of your nzyme-node configuration file lets you fine-tune certain parameters that will
influece the resource usage and performance of your Nzyme nodes.
Report Processing
Reports sent by taps can contain a lot of data and lead to a lot of parsing and database inserts. This process is multithreaded by default, and you can increase the number of threads to speed up processing.
It is important to keep total number of available threads/cores on your CPUs in mind when setting a parallel processing configuration like this.
# Performance tuning.
performance: {
# How many threads work on processing incoming tap reports.
report_processor_pool_size: 3
...
}
Tuning nzyme-tap
CPU Threads
The performance section of your nzyme-tap configuration file contains variables for how many parallel threads
the taps will spin up for data processing.
Assuming nzyme-tap is the only resource-intensive process running on the host, you should aim to configure as many
parallel Nzyme threads as your CPU can run, minus at least one thread reserved for the operating system and other Nzyme
tasks. Configuring more Nzyme threads than your CPU can run in parallel will not lead to an error but can negatively
affect performance.
Thread and Processor Model
Nzyme is spreading commonly high-throughput data types like TCP or UDP across dedicated thread pools. Other data, like Bluetooth or UAV information, is routed into a shared thread pool.
This design allows you to quickly tune thread pools without too many independent configuration variables.
The ethernet_broker and wifi_broker thread count configuration can almost always be left at the default of 1.
The brokers are responsible for routing data from captures to their respective thread pools and should be extremely
fast and efficient. You should only increase broker threads if you see capture errors.
Example Configuration
For example, if your CPU can run 12 threads in parallel (6 cores with HyperThreading) and you are collecting a lot of Ethernet data, you could configure the following:
[performance]
ethernet_brokers = 1
wifi_brokers = 1
...
[protocols.wifi]
pipeline_size = 16384
processors = 1
...
[protocols.tcp]
pipeline_size = 16384
processors = 4
...
[protocols.udp]
pipeline_size = 16384
processors = 3
...
This will give Nzyme 4 threads to process TCP data, 3 for UDP data, 1 to process WiFi data, 2 for brokers and still leave two threads for everything else.
WiFi traffic tends to be very low, and you will often reach sufficient performance with a single thread.
There is no one single formula, because hardware, environment and data can be so different from setup to setup. It is recommended to play around with these parameters after making a first best guess.
Buffer and Pipeline Sizes
One goal of nzyme-tap is to move recorded data out of the capture handle and into its own processing logic. This way,
the data queuing is under our control and no packets are lost in case of a backlog. To accomplish this, nzyme-tap
only performs very rudimentary packet parsing before moving the data into an in-memory buffer. Brokers are reading
data from that buffer as fast as they can, perform more parsing and submit the parsed data into channels for processing.
The default broker buffer sizes are fairly large and assume that the tap is not running on an extremely memory-constraint environment like, for example, embedded systems. In such a case, lowering the buffer capacities can be helpful.
Another reason to lower buffer capacities is to avoid memory exhaustion from extremely large packets. If you receive huge packets, even a small amount of them could cause issues by running out of memory.
A reason to raise the buffer capacities would be to increase parallel throughput of packets. If you are in an extremely high-throughput environment, the default buffers might be simply too small to keep all packets in them before they are processed.
In case of dropped packets, you will see warnings in the nzyme-tap log file as well as on the taps details page. In
addition, a TAP_DROP or TAP_BUFFER health indicator will be triggered and subscribed actions executed.
The broker buffer size configuration looks like this:
Individual data and protocol type buffer size configuration looks like this: