NFD

I'm having a hard time understanding what this issue is about. The beginning of the description is pretty general and it sounds like you're talking about the whole forwarder... then you mention the best-route strategy (so, is it only about best-route?)... and right after you mentioned self-learning (and now you lost me)... please clarify.

the forwarding plane should be able to try other next hops if there are any

It is already able to do that. What and when other nexthops are tried is up to the strategy.

The best-route strategy has always had pretty specific forwarding semantics. One of its most important characteristics is that it only looks at routes in the FIB, starting from the lowest-cost nexthop. Introducing a new version of best-route with completely different semantics is not a good idea in my opinion. We have a virtually infinite set of strategy names to choose from, why do you want to overload the already poorly named best-route?

Davide Pesavento wrote:

The best-route strategy has always had pretty specific forwarding semantics. One of its most important characteristics is that it only looks at routes in the FIB, starting from the lowest-cost nexthop. Introducing a new version of best-route with completely different semantics is not a good idea in my opinion. We have a virtually infinite set of strategy names to choose from, why do you want to overload the already poorly named best-route?

I don't care much about the strategy name. The point is whatever name we choose, we need to make it the default strategy for / in the conf file. The current default strategy is best route, so I used it. Do you have a name suggestion?

Teng Liang wrote:

I don't care much about the strategy name.

Names are important. We've already gotten enough complaints about the "best route" name for the current default strategy, because it doesn't clearly express the semantics. Let's not make it even worse.

Do you have a name suggestion?

Ehm... what's wrong with "self-learning"? Really, this issue should be about improving/fixing the existing self-learning strategy to handle more (real-world) cases. There is no need to merge anything with anything. Have a properly working, robust, self-learning strategy (as it should have been from the beginning) and then we can consider changing the default in nfd.conf.

Teng Liang wrote:

• If multiple next hops exist, an eligible next hop with the lowest cost should be picked. For a retransmitted Interest that can be forwarded, forward it to an unused next hop with the lowest cost; if all next hops have been tried, pick the first chosen next hop to forward.

How do you assign costs to routes created through self-learning?

1. Once all next hops return no-route NACK
   • if this node is consumer, reinitiate discovery Interest flooding
   • if this node is not consumer, send the NACK back to downstreams

How do you plan to implement the "consumer or not?" check?

• If the same retransmitted Interest has been forwarded 5 times (RETX_TRIGGER_BROADCAST_COUNT), reinitiate Interest flooding.

Who would do this? Any node in the network?

Davide Pesavento wrote:

Teng Liang wrote:

• If multiple next hops exist, an eligible next hop with the lowest cost should be picked. For a retransmitted Interest that can be forwarded, forward it to an unused next hop with the lowest cost; if all next hops have been tried, pick the first chosen next hop to forward.

How do you assign costs to routes created through self-learning?

A hard-coded value (2048 for the current imp).

1. Once all next hops return no-route NACK
   • if this node is consumer, reinitiate discovery Interest flooding
   • if this node is not consumer, send the NACK back to downstreams

How do you plan to implement the "consumer or not?" check?

If there is one in-record, the face is local.

• If the same retransmitted Interest has been forwarded 5 times (RETX_TRIGGER_BROADCAST_COUNT), reinitiate Interest flooding.

Who would do this? Any node in the network?

Should be the consumer node only (will update the processing flow).

Teng Liang wrote:

Davide Pesavento wrote:

How do you assign costs to routes created through self-learning?

A hard-coded value (2048 for the current imp).

And how do you choose the next hop when you have multiple routes discovered via self-learning? In other words, how do you rank next hops with the same cost?

How do you plan to implement the "consumer or not?" check?

If there is one in-record, the face is local.

Huh? Having one in-record doesn't mean that the face is local. And not all consumers are connected to NFD through a local face.

Davide Pesavento wrote:

Teng Liang wrote:

Davide Pesavento wrote:

How do you assign costs to routes created through self-learning?

A hard-coded value (2048 for the current imp).

And how do you choose the next hop when you have multiple routes discovered via self-learning? In other words, how do you rank next hops with the same cost?

The current self-learning will only learn one route (the first coming back Data with PrefixAnn). It is possible to learn multiple paths in future designs, but how to set the cost is undefined yet.

How do you plan to implement the "consumer or not?" check?

If there is one in-record, the face is local.

Huh? Having one in-record doesn't mean that the face is local. And not all consumers are connected to NFD through a local face.

True. First, I meant if there is only one in-record, and the inFace is local. Second, these conditions cover our testing scenarios. Do your know if there's other information can be used to make better decision?

Teng Liang wrote:

Davide Pesavento wrote:

Teng Liang wrote:

Davide Pesavento wrote:

How do you assign costs to routes created through self-learning?

A hard-coded value (2048 for the current imp).

And how do you choose the next hop when you have multiple routes discovered via self-learning? In other words, how do you rank next hops with the same cost?

The current self-learning will only learn one route (the first coming back Data with PrefixAnn). It is possible to learn multiple paths in future designs, but how to set the cost is undefined yet.

So once you've learned one route/nexthop, you never flood discovery Interests anymore?

Huh? Having one in-record doesn't mean that the face is local. And not all consumers are connected to NFD through a local face.

True. First, I meant if there is only one in-record, and the inFace is local. Second, these conditions cover our testing scenarios. Do your know if there's other information can be used to make better decision?

Can you avoid basing the "re-flood" decision on whether it's a consumer node or not? It seems fragile in general. Maybe you can let the consumer app (or library) itself retransmit the Interest, and base the re-flood logic on incoming retransmissions?

Davide Pesavento wrote:

Teng Liang wrote:

Davide Pesavento wrote:

Teng Liang wrote:

Davide Pesavento wrote:

How do you assign costs to routes created through self-learning?

A hard-coded value (2048 for the current imp).

And how do you choose the next hop when you have multiple routes discovered via self-learning? In other words, how do you rank next hops with the same cost?

The current self-learning will only learn one route (the first coming back Data with PrefixAnn). It is possible to learn multiple paths in future designs, but how to set the cost is undefined yet.

So once you've learned one route/nexthop, you never flood discovery Interests anymore?

If there is a no-route NACK (and after tried all next hops), or the Interest retransmission (not suppressed) has reached a threshold. Re-flooding Interests only happens at consumer.

Huh? Having one in-record doesn't mean that the face is local. And not all consumers are connected to NFD through a local face.

True. First, I meant if there is only one in-record, and the inFace is local. Second, these conditions cover our testing scenarios. Do your know if there's other information can be used to make better decision?

Can you avoid basing the "re-flood" decision on whether it's a consumer node or not? It seems fragile in general. Maybe you can let the consumer app (or library) itself retransmit the Interest, and base the re-flood logic on incoming retransmissions?

Why is it fragile, is it because it is hard to decide if the the NFD is directly connecting to consumer? Depending on apps may not be reliable. E.g., the current ndn catchunk just terminates on receiving a no-route NACK.

Teng Liang wrote:

Davide Pesavento wrote:

Teng Liang wrote:

The current self-learning will only learn one route (the first coming back Data with PrefixAnn). It is possible to learn multiple paths in future designs, but how to set the cost is undefined yet.

So once you've learned one route/nexthop, you never flood discovery Interests anymore?

If there is a no-route NACK (and after tried all next hops), or the Interest retransmission (not suppressed) has reached a threshold.

This "if" is missing a "then" branch, so I don't understand the sentence.

Re-flooding Interests only happens at consumer.

And what if the re-flooding discovers a new nexthop? This is what I asked earlier and you said "The current self-learning will only learn one route"... you seem to be contradicting yourself.

Can you avoid basing the "re-flood" decision on whether it's a consumer node or not? It seems fragile in general. Maybe you can let the consumer app (or library) itself retransmit the Interest, and base the re-flood logic on incoming retransmissions?

Why is it fragile, is it because it is hard to decide if the the NFD is directly connecting to consumer?

Yes, it's impossible in the general case.

Depending on apps may not be reliable.

You said previously that when the number of retransmissions reaches a threshold, you trigger re-flooding. So you are relying on apps... which contradicts the above statement.

E.g., the current ndn catchunk just terminates on receiving a no-route NACK.

I don't get this argument. If catchunks doesn't behave properly, fix it! instead of adding workarounds to the forwarder.

Teng Liang wrote:

because on-demand face cannot be created directly by protocol factory

This is a purely implementational concern and should not affect any design decision.

and unicast faces learned in local environment are indeed persistent.

What does this mean? Sounds like a circular argument. Please clarify.

I'm a little concerned about keeping the auto-created faces "forever". There is clearly a potential resource consumption issue here (there is a per-process limit on the number of open file descriptors), and it's easily exploitable.

Davide Pesavento wrote:

Teng Liang wrote:

because on-demand face cannot be created directly by protocol factory

This is a purely implementational concern and should not affect any design decision.

Right, so what was the concern for to avoid on-demand face creation in protocol factory?

and unicast faces learned in local environment are indeed persistent.

What does this mean? Sounds like a circular argument. Please clarify.

I'm a little concerned about keeping the auto-created faces "forever". There is clearly a potential resource consumption issue here (there is a per-process limit on the number of open file descriptors), and it's easily exploitable.

The number of unicast faces in a local network is normally limited, and their existence are consistent. But I accept either way.

Teng Liang wrote:

Davide Pesavento wrote:

Teng Liang wrote:

because on-demand face cannot be created directly by protocol factory

This is a purely implementational concern and should not affect any design decision.

Right, so what was the concern for to avoid on-demand face creation in protocol factory?

No concern. There was simply no use case.

Would you still be interested in adding this feature to the NFD codebase? A colleague of mine and I updated Teng's NFD fork a while ago to the latest upstream state, so we could potentially follow up on the work that has already been done and, for example, incorporate the review comments made to the changes/patch sets pending on Gerrit (such as https://gerrit.named-data.net/c/NFD/+/6230/1).

Jan Romann wrote in #note-36:

Would you still be interested in adding this feature to the NFD codebase? A colleague of mine and I updated Teng's NFD fork a while ago to the latest upstream state, so we could potentially follow up on the work that has already been done and, for example, incorporate the review comments made to the changes/patch sets pending on Gerrit (such as https://gerrit.named-data.net/c/NFD/+/6230/1).

Yes, we are interested! The current NFD implements part of self-learning but not the full mechanism due to certain issues. It may be more effective to discuss this over a zoom meeting if you're interested. We have weekly meeting on Friday 9-11am Pacific Time and you're welcome to join. We'd like to learn more about how self-learning fits into your research and how your code will address certain complexity in implementation. But if the meeting time is not good for you, maybe you can follow up on Gerrit to get this going. Thanks!

Beichuan Zhang wrote in #note-37:

Yes, we are interested! The current NFD implements part of self-learning but not the full mechanism due to certain issues. It may be more effective to discuss this over a zoom meeting if you're interested. We have weekly meeting on Friday 9-11am Pacific Time and you're welcome to join. We'd like to learn more about how self-learning fits into your research and how your code will address certain complexity in implementation. But if the meeting time is not good for you, maybe you can follow up on Gerrit to get this going. Thanks!

That is great, thank you very much for your response :) My colleague and I will probably join you in today's call, looking forward to discussing this topic with you!