Project

General

Profile

Feature #4973

Self-learning: switch from multicast to unicast on Wi-Fi infrastructure

Added by Teng Liang 10 months ago. Updated 3 days ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Forwarding
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:

Description

The current self-learning implementation of #4279 allows NFDs to learn FIB entries through Prefix Announcement piggybacked on Data replying to discovery Interest. The learned next hop is where Data sent back, so the next hop will be a multicast Face by default.

We tested the performance of UDP multicast and unicast Face in WiFi AP-station mode. The testing topology consisted of two laptops connecting to one Openwrt router.NDN catchunks and putchunks were used to download and publish data. The goodput of unicast is about 5x as good as the multicast. The detailed statistics is listed in the end. Therefore, the multicast/Unicast face switching is important.

To implement Face switching, our existing effort was to extend the concept of Face to Face-EndpointId. This approach will leave EndpointId in table, management and forwarder modules, but these modules in NDN should not deal with EndpointId.

To avoid Face extension with EndpointId, this design allows forwarding strategy to create unicast Face on receiving Data packet from multicast Face. The implementation design has 7 steps described below.

Step 1
Pass face system to forwarder, so forwarder can use it to create face for strategies.

Step 2
To create a face, face system has to know which protocol factory should be used, and required parameters, including local uri, remote uri and parameters related to face.

  • To get ProtocolFactory, one way is to add ProtocolFactory Id to Transport, then FaceSystem is able to get ProtocolFactory from Face->Transport->ProtocolFactory Id
  • The Remote Uri can be constructed from EndpointId and ProtocolFactoryId.

Step 3: strategy requests unicast face creation

void
SelfLearningStrategy::afterReceiveData(const shared_ptr<pit::Entry>& pitEntry, const FaceEndpoint& ingress, const Data& data)
{
     // 1. check if the unicast face needs to be created
     // 2, prolong PIT entry lifetime, so that PIT entry still exists when face is created
     // 3. request face creation 
     this->createUnicastFaceOnData(pitEntry, data);
}

Step 4: Forwarder forwards face creation request to FaceSystem

void
Forwarder::createUnicastFaceOnData(const PitEntry& pitEntry, const Data& data)
{
    faceSystem.createUnicastFaceOnData(data, [] (const Face& face) {
        // Step 5

        // 1. if PIT entry has been deleted, return
        // 2. find effective strategy for pitEntry.getName()
        // 3. strategy.afterUnicastFaceCreation(pitEntry, face)
    });
}

Step 5: FaceSystem asks ProtocolFactory to create face

void
FaceSystem::createUnicastFaceOnData(const Data& data, function<void(const Face& face)> callback)
{
    // 1. extract packet tag and determine which ProtocolFactory owns the multicast face
    // 2. pass remote endpoint information to the ProtocolFactory, which then creates the face
    ProtocolFactory.createFace(remoteEndpoint, callback);
}

Step 6 is in the callback function in the step three.

Step 7: use the newly created face in the callback function

void
SelfLearningStrategy::afterUnicastFaceCreation(const PitEntry& pitEntry, const Face& face)
{
  // add route towards the face
}

class Strategy {
public:
    void afterUnicastFaceCreation(const PitEntry& pitEntry, const Face& face);

protected:
    void createUnicastFaceOnData(const PitEntry& pitEntry, const Data& data);
};

NDN catchunks/putchanks for consumer---WiFi AP---producer

Unicast:

All segments have been received.
Time elapsed: 6.83123 seconds
Segments received: 1317
Transferred size: 1447.75 kB
Goodput: 1.695452 Mbit/s
Congestion marks: 0 (caused 0 window decreases)
Timeouts: 23 (caused 16 window decreases)
Retransmitted segments: 23 (1.71642%), skipped: 0
RTT min/avg/max = 8.058/71.200/421.579 ms

Multicast:

All segments have been received.
Time elapsed: 33.6946 seconds
Segments received: 1317
Transferred size: 1447.75 kB
Goodput: 343.734634 kbit/s
Congestion marks: 0 (caused 0 window decreases)
Timeouts: 553 (caused 191 window decreases)
Retransmitted segments: 550 (29.459%), skipped: 3
RTT min/avg/max = 16.970/88.583/333.864 ms


Related issues

Blocked by NFD - Task #5041: Redefine EndpointId as variant<ethernet::Address, ip::udp::endpoint>NewTeng Liang

Actions
#1

Updated by Davide Pesavento 10 months ago

  • Tags set to SelfLearning
  • Subject changed from self-learning strategy: switching from multicast to unicast on WiFi infrastructure to Self-learning: switching from multicast to unicast on WiFi infrastructure
  • Assignee set to Teng Liang
#2

Updated by Teng Liang 10 months ago

As discussed in the NFD call on July 31, 2019, the third design (with some changes) involves the minimum codebase update. More discussions are needed on next Monday's NFD call.

Here is the basic workflow of how self-learning works with face switching:

  1. A consumer multicasts "discovery" Interests to all potential faces, if the consumer has no routes.
  2. On receiving Data packets, self-learning checks their incoming Face.
  • if the Face is unicast, self-learning add routes towards the Face (with the same behaviors as in #4279)
  • if the Face is multicast, self-learning creates a unicast face towards the Data sender, and adds routes routes towards the unicast Face.

In this design, face creation is at consumers, and the first Interest-Data exchange is symmetric.

The main task is to create unicast face on receiving data from multicast face.

#3

Updated by Teng Liang 9 months ago

To allow fw strategy create a unicast UDP face from a multicast UDP face in NFD, here are the tasks:

  1. in Nfd class, add a static method getFaceManager to access the FaceManager instance
  2. in FaceManager class, add a method createUnicastFromMulticastFace, that accepts three parameters FaceId, onSuccessCallback, and onFailureCallback.
  • In this method, the unicast face parameters are obtained from the multicast face, that is found by FaceId and Face::getTransport
  • After getting all necessary information, try to create the unicast face, if success, invoke onSuccessCallback with the new unicast Face as the parameter, otherwise invoke onFailureCallback with the multicast Face as the parameter.

Update in self-learning fw strategy:

  • the addRoute method needs to moved to the callback functions
#4

Updated by Davide Pesavento 9 months ago

Teng Liang wrote:

  1. in Nfd class, add a static method getFaceManager to access the FaceManager instance

Don't like this. Static stuff is evil (in most cases) and makes dependency injection a lot harder if not impossible. Why can't you pass a FaceManager& to the objects that need it? (but also see below)

  1. in FaceManager class, add a method createUnicastFromMulticastFace, that accepts three parameters FaceId, onSuccessCallback, and onFailureCallback.

You need to explain this point in a lot more detail... face creation triggers a number of other things, you need to convince us that this is all going to work out, and for that you need to be more specific.

Also, you haven't justified why you need to go through management to create the face.

In this method, the unicast face parameters are obtained from the multicast face, that is found by FaceId and Face::getTransport

What parameters exactly? How will you initialize those parameters that make sense only for unicast faces?

#5

Updated by Teng Liang 9 months ago

Davide Pesavento wrote:

Also, you haven't justified why you need to go through management to create the face.

Looking into the details of FaceManager::createFace, it is not necessary to go though FaceManager in this case. Fw only needs to accessFaceTable and FaceSystem for face creation.

Then, the question is how to pass the FaceManager object in Nfd class to the forwarder object. One option is to make m_faceSystem shared_ptr, and call forwarder.setFaceSystem after it is initialized. However, this way does not seem elegant enough.

#6

Updated by Teng Liang 8 months ago

As discussed in the NFD call on Oct. 2 and in another call with Junxiao, we nailed down the implementation design details:

STEP ONE
When packet arrives at a multicast face, the packet is tagged:

  • which ProtocolFactory owns the multicast face?
  • what's the remote endpoint? (format defined within ProtocolFactory)

STEP TWO: strategy requests unicast face creation

void
MyStrategy::processData(const PitEntry& pitEntry, const Data& data)
{
    // 1. prolong PIT entry lifetime, so that PIT entry still exists when face is created

    // 2. request face creation
    this->createUnicastFaceForPacketSender(pitEntry, data);
}

STEP THREE: forwarding requests face creation from Forwarder to FaceTable

void
Forwarder::createUnicastFaceForPacketSender(const PitEntry& pitEntry, const Packet& pkt)
{
    faceTable.createUnicastFaceForPacketSender(pkt, [] (const Face& face) {
        // STEP FIVE

        // 1. if PIT entry has been deleted, return
        // 2. find effective strategy for pitEntry.getName()
        // 3. strategy.afterUnicastFaceCreation(pitEntry, face)
    });
}
class FaceTable
{
public:
    signal::Signal<FaceTable, Packet, function<void(const Face& face)>> requestCreateUnicastFaceForPacketSender;

    void
    createUnicastFaceForPacketSender(const Packet& pkt, function<void(const Face& face)> callback)
    {
        requestCreateUnicastFaceForPacketSender(pkt, callback);
    }
};

STEP FOUR: FaceSystem asks ProtocolFactory to create face

  • note : this is hooked to faceTable.requestCreateUnicastFaceForPacketSender signal
void
FaceSystem::createUnicastFaceForPacketSender(const Packet& pkt, function<void(const Face& face)> callback)
{
    // 1. extract packet tag and determine which ProtocolFactory owns the multicast face
    // 2. pass remote endpoint information to the ProtocolFactory, which then creates the face
    ProtocolFactory.createFace(remoteEndpoint, callback);
}

STEP FIVE is in the callback function in the step three.

STEP SIX: use the newly created face in the callback function

void
MyStrategy::afterUnicastFaceCreation(const PitEntry& pitEntry, const Face& face)
{
}

class Strategy {
public:
    void afterUnicastFaceCreation(const PitEntry& pitEntry, const Face& face);

protected:
    void createUnicastFaceForPacketSender(const PitEntry& pitEntry, const Packet& pkt);
};
#7

Updated by Davide Pesavento 8 months ago

  • Category set to Forwarding
  • Start date deleted (07/29/2019)

Firstly, I completely disagree with using a signal to perform an action. Signals are for notifications. Moreover, it adds another unnecessary layer of indirection and additional complexity. Just use FaceSystem to create the face.

Secondly, given that after face creation you need to register a route on that face (which is also async), maybe the two operations can be combined and the strategy be re-invoked only once at the end of the combined operation?

#8

Updated by Teng Liang 8 months ago

An alternative design is to pass FaceSystem to Forwarder. This is simpler as Davide mentioned. The downside is that FaceSystem has to be created for Forwarder unit testing, which is acceptable to me

STEP ONE
When Data arrives at a multicast face, the Data is tagged:

  • the ProtocolFactory that owns the multicast face
  • the remote uri (format defined within ProtocolFactory)

STEP TWO: strategy requests unicast face creation

void
SelfLearningStrategy::afterReceiveData(const shared_ptr<pit::Entry>& pitEntry, const FaceEndpoint& ingress, const Data& data)
{
     // 1. check if the unicast face needs to be created
     // 2, prolong PIT entry lifetime, so that PIT entry still exists when face is created
     // 3. request face creation 
     this->createUnicastFaceOnData(pitEntry, data);
}

STEP THREE: Forwarder forwards face creation request to FaceSystem

void
Forwarder::createUnicastFaceOnData(const PitEntry& pitEntry, const Data& data)
{
    faceSystem.createUnicastFaceOnData(data, [] (const Face& face) {
        // STEP FIVE

        // 1. if PIT entry has been deleted, return
        // 2. find effective strategy for pitEntry.getName()
        // 3. strategy.afterUnicastFaceCreation(pitEntry, face)
    });
}

STEP FOUR: FaceSystem asks ProtocolFactory to create face

void
FaceSystem::createUnicastFaceOnData(const Data& data, function<void(const Face& face)> callback)
{
    // 1. extract packet tag and determine which ProtocolFactory owns the multicast face
    // 2. pass remote endpoint information to the ProtocolFactory, which then creates the face
    ProtocolFactory.createFace(remoteEndpoint, callback);
}

STEP FIVE is in the callback function in the step three.

STEP SIX: use the newly created face in the callback function

void
SelfLearningStrategy::afterUnicastFaceCreation(const PitEntry& pitEntry, const Face& face)
{
  // add route towards the face
}

class Strategy {
public:
    void afterUnicastFaceCreation(const PitEntry& pitEntry, const Face& face);

protected:
    void createUnicastFaceOnData(const PitEntry& pitEntry, const Data& data);
};
#9

Updated by Junxiao Shi 8 months ago

Signals are for notifications.

False. Face type has signals to start forwarding pipeline processing of incoming packets.

Moreover, it adds another unnecessary layer of indirection and additional complexity. Just use FaceSystem to create the face.

Using FaceSystem causes significant complexity in isolating forwarding for unit testing.
Unless, you define BaseFaceSystem as an abstract class with no implementation, and provide a complete mock alongside the real thing. Bottom line is, forwarding unit testing cannot rely on any of the protocol factories.

after face creation you need to register a route on that face (which is also async), maybe the two operations can be combined and the strategy be re-invoked only once at the end of the combined operation?

Yes.

#10

Updated by Davide Pesavento 8 months ago

Junxiao Shi wrote:

Signals are for notifications.

False. Face type has signals to start forwarding pipeline processing of incoming packets.

Nope. You're confusing the signal with its handlers. Face's signals merely notify of the arrival of a packet, what happens after is defined by the handler (or handlers), not by the signal.

Moreover, it adds another unnecessary layer of indirection and additional complexity. Just use FaceSystem to create the face.

Using FaceSystem causes significant complexity in isolating forwarding for unit testing.

Complexity in unit tests is preferable to complexity in the actual forwarder. Tests come second.

Unless, you define BaseFaceSystem as an abstract class with no implementation, and provide a complete mock alongside the real thing.

Which would be the proper testing procedure anyway. We hardly mock anything in our current unit tests out of laziness. Besides, in order to create a face, Forwarder will only need one or two functions from FaceSystem (depending on how Teng designs this part, but it's a minor detail), so you won't need to abstract the entire FaceSystem class and you won't need a "complete mock". It's really quite simple in fact.

#11

Updated by Junxiao Shi 8 months ago

Complexity in unit tests is preferable to complexity in the actual forwarder. Tests come second.

No. Test coverage is important for software quality.

Unless, you define BaseFaceSystem as an abstract class with no implementation, and provide a complete mock alongside the real thing.

Which would be the proper testing procedure anyway. We hardly mock anything in our current unit tests out of laziness. Besides, in order to create a face, Forwarder will only need one or two functions from FaceSystem (depending on how Teng designs this part, but it's a minor detail), so you won't need to abstract the entire FaceSystem class and you won't need a "complete mock". It's really quite simple in fact.

I'm talking more about non-self-learning tests. They should not require creation of the real FaceSystem along with all the protocol factories.

#12

Updated by Davide Pesavento 8 months ago

Junxiao Shi wrote:

Complexity in unit tests is preferable to complexity in the actual forwarder. Tests come second.

No. Test coverage is important for software quality.

https://yourlogicalfallacyis.com/strawman

None of what I suggested above has the effect of reducing test coverage.

#13

Updated by Teng Liang 7 months ago

STEP one in note-8 does not describe implementation details. The target is to get ProtocolFactory and RemoteUri when calling functions in FaceSystem, in order to create a unicast UDP/Ether face.

  • To get ProtocolFactory, one way is to add ProtocolFactory Id to Transport, then FaceSystem is able to get ProtocolFactory from Face->Transport->ProtocolFactory Id
  • The Remote Uri can be attached to Data by the sender. This should only happen to discovery Interest sent to multicast Face.
#14

Updated by Junxiao Shi 7 months ago

note-13 design makes sense.

#15

Updated by Teng Liang 7 months ago

  • Description updated (diff)
#16

Updated by Teng Liang 7 months ago

  • Description updated (diff)
#17

Updated by Davide Pesavento 7 months ago

Teng Liang wrote:

  • The Remote Uri can be attached to Data by the sender. This should only happen to discovery Interest sent to multicast Face.

I strongly disagree with this for a number of reasons. But we already discussed another (better) approach at today's call so I don't need to elaborate.

#18

Updated by Teng Liang 7 months ago

  • Description updated (diff)

Davide Pesavento wrote:

Teng Liang wrote:

  • The Remote Uri can be attached to Data by the sender. This should only happen to discovery Interest sent to multicast Face.

I strongly disagree with this for a number of reasons. But we already discussed another (better) approach at today's call so I don't need to elaborate.

Yes, the description is updated.

#19

Updated by Alex Afanasyev 6 days ago

  • Blocked by Task #5041: Redefine EndpointId as variant<ethernet::Address, ip::udp::endpoint> added
#20

Updated by Davide Pesavento 3 days ago

  • Subject changed from Self-learning: switching from multicast to unicast on WiFi infrastructure to Self-learning: switch from multicast to unicast on Wi-Fi infrastructure
  • Status changed from New to In Progress

Also available in: Atom PDF