Feature #5059
open
Congestion Detection: Monitor NIC incoming buffer queue
0%
Description
I noticed in VM experiments that often NFD processing speed is the performance bottleneck, which leads to many/all of packet loss being caused by drops at a router's incoming queue.
One solution would be to monitor the incoming queue, and if it exceeds some threshold to put in a congestion mark to tell the consumers to slow down.
The relevant buffer information can be queried with the "ss" (socket status) tool, although only in the version available in Ubuntu 18.04 and later.
ss -u -m
Recv-Q Send-Q Local Address:Port Peer Address:Port
0 0 10.1.0.1:6363 10.2.0.3:6363
skmem:(r20352,rb212992,t0,tb212992,f128,w0,o0,bl0,d2549)
The relevant information here are the 2549 socket drops.
-m, --memory
Show socket memory usage. The output format is:
skmem:(r<rmem_alloc>,rb<rcv_buf>,t<wmem_alloc>,tb<snd_buf>,
f<fwd_alloc>,w<wmem_queued>,o<opt_mem>,
bl<back_log>,d<sock_drop>)
We could query this information with the netlink API, however it is quite complicated to use.
For example, the ss tool itself uses this API, and takes up over 5000 lines of code. https://github.com/shemminger/iproute2/blob/master/misc/ss.c
Updated by Junxiao Shi over 4 years ago
often NFD processing speed is the performance bottleneck
In a local area network, this occurs more than often.
if the incoming queue exceeds some threshold to put in a congestion mark
What's the algorithm for congestion signalling? Can this use the same "sojourn time" logic like the egress queue, or does this need a different logic?
Updated by Anonymous over 4 years ago
Junxiao Shi wrote:
if the incoming queue exceeds some threshold to put in a congestion mark
What's the algorithm for congestion signalling? Can this use the same "sojourn time" logic like the egress queue, or does this need a different logic?
Good question. Certainly it makes sense to try out the CoDel sojourn time logic for the current queue size (w).
Moreover, I think it's a good idea to put in one congestion mark for every dropped packet, in order to signal faster than waiting for a timeout.
Lastly, if the marked packet would be in Interest, one should instead note this in the PIT entry and only mark the returning Data packet. This logic is described in the PCON paper https://named-data.net/publications/practical_congestion_control_scheme/
Updated by Davide Pesavento over 4 years ago
- Category set to Faces
- Start date deleted (
11/27/2019)
Updated by Davide Pesavento over 4 years ago
Klaus Schneider wrote:
although only in the version available in Ubuntu 18.04 and later
SK_MEMINFO_DROPS was added in Linux 4.7
We could query this information with the netlink API, however it is quite complicated to use.
For example, the ss tool itself uses this API, and takes up over 5000 lines of code. https://github.com/shemminger/iproute2/blob/master/misc/ss.c
Well, netlink can be tricky, yes, but 5000 lines are for the entire ss tool. We just need SK_MEMINFO_DROPS, which should "only" take a few hundreds lines. And ndn-cxx already has some infrastructure to deal with netlink sockets and messages (used by NetworkMonitor) which hopefully can be reused.
Updated by Davide Pesavento over 4 years ago
Apparently, a few kernel versions later, an (undocumented?) socket option was introduced. It returns the same information without having to deal with netlink (and it's only one syscall).
Updated by Anonymous over 4 years ago
Davide Pesavento wrote:
Klaus Schneider wrote:
although only in the version available in Ubuntu 18.04 and later
SK_MEMINFO_DROPSwas added in Linux 4.7
Only counting Ubuntu LTS versions, what I said is still correct :)
Yeah, you can get the feature in Ubuntu 16.10 -- 17.10 as well, but they are no longer supported.
Apparently, a few kernel versions later, an (undocumented?) socket option was introduced -- https://patchwork.ozlabs.org/patch/741147/. It > returns the same information without having to deal with netlink (and it's only one syscall).
This looks promising. Could make implementing it much easier.
Suggested by Eric Dumazet.
FYI This is the same guy who implemented the CoDel AQM in the Linux kernel.
Updated by Davide Pesavento 4 months ago
- Tags set to needs-discussion
- Parent task deleted (
#1624)
Updated by Davide Pesavento 4 months ago
- Related to Feature #1624: Design and Implement Congestion Control added