Project

General

Profile

Feature #5059

Feature #1624: Design and Implement Congestion Control

Congestion Detection: Monitor NIC incoming buffer queue

Added by Klaus Schneider about 2 months ago. Updated about 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Faces
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:

Description

I noticed in VM experiments that often NFD processing speed is the performance bottleneck, which leads to many/all of packet loss being caused by drops at a router's incoming queue.

One solution would be to monitor the incoming queue, and if it exceeds some threshold to put in a congestion mark to tell the consumers to slow down.

The relevant buffer information can be queried with the "ss" (socket status) tool, although only in the version available in Ubuntu 18.04 and later.

ss -u -m
Recv-Q     Send-Q            Local Address:Port           Peer Address:Port
0          0                      10.1.0.1:6363               10.2.0.3:6363
        skmem:(r20352,rb212992,t0,tb212992,f128,w0,o0,bl0,d2549)

The relevant information here are the 2549 socket drops.

       -m, --memory
              Show socket memory usage. The output format is:

              skmem:(r<rmem_alloc>,rb<rcv_buf>,t<wmem_alloc>,tb<snd_buf>,
                            f<fwd_alloc>,w<wmem_queued>,o<opt_mem>,
                            bl<back_log>,d<sock_drop>)

We could query this information with the netlink API, however it is quite complicated to use.

For example, the ss tool itself uses this API, and takes up over 5000 lines of code. https://github.com/shemminger/iproute2/blob/master/misc/ss.c

History

#1

Updated by Junxiao Shi about 2 months ago

often NFD processing speed is the performance bottleneck

In a local area network, this occurs more than often.

if the incoming queue exceeds some threshold to put in a congestion mark

What's the algorithm for congestion signalling? Can this use the same "sojourn time" logic like the egress queue, or does this need a different logic?

#2

Updated by Klaus Schneider about 2 months ago

Junxiao Shi wrote:

if the incoming queue exceeds some threshold to put in a congestion mark

What's the algorithm for congestion signalling? Can this use the same "sojourn time" logic like the egress queue, or does this need a different logic?

Good question. Certainly it makes sense to try out the CoDel sojourn time logic for the current queue size (w).

Moreover, I think it's a good idea to put in one congestion mark for every dropped packet, in order to signal faster than waiting for a timeout.

Lastly, if the marked packet would be in Interest, one should instead not this in the PIT entry and only mark the returning Data packet. This logic is described in the PCON paper https://named-data.net/publications/practical_congestion_control_scheme/

#3

Updated by Davide Pesavento about 2 months ago

  • Category set to Faces
  • Start date deleted (11/27/2019)
#4

Updated by Davide Pesavento about 2 months ago

Klaus Schneider wrote:

although only in the version available in Ubuntu 18.04 and later

SK_MEMINFO_DROPS was added in Linux 4.7

We could query this information with the netlink API, however it is quite complicated to use.

For example, the ss tool itself uses this API, and takes up over 5000 lines of code. https://github.com/shemminger/iproute2/blob/master/misc/ss.c

Well, netlink can be tricky, yes, but 5000 lines are for the entire ss tool. We just need SK_MEMINFO_DROPS, which should "only" take a few hundreds lines. And ndn-cxx already has some infrastructure to deal with netlink sockets and messages (used by NetworkMonitor) which hopefully can be reused.

#5

Updated by Davide Pesavento about 2 months ago

Apparently, a few kernel versions later, an (undocumented?) socket option was introduced. It returns the same information without having to deal with netlink (and it's only one syscall).

#6

Updated by Klaus Schneider about 2 months ago

Davide Pesavento wrote:

Klaus Schneider wrote:

although only in the version available in Ubuntu 18.04 and later

SK_MEMINFO_DROPS was added in Linux 4.7

Only counting Ubuntu LTS versions, what I said is still correct :)

Yeah, you can get the feature in Ubuntu 16.10 -- 17.10 as well, but they are no longer supported.

Apparently, a few kernel versions later, an (undocumented?) socket option was introduced -- https://patchwork.ozlabs.org/patch/741147/. It > returns the same information without having to deal with netlink (and it's only one syscall).

This looks promising. Could make implementing it much easier.

Suggested by Eric Dumazet.

FYI This is the same guy who implemented the CoDel AQM in the Linux kernel.

Also available in: Atom PDF