Project

General

Profile

Feature #4290

Give strategy authority over Data

Added by Junxiao Shi about 1 year ago. Updated 6 months ago.

Status:
Closed
Priority:
High
Assignee:
Category:
Forwarding
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
6.00 h

Description

Current NDN protocol allows a single Data to satisfy multiple pending Interests. In NFD, each pending Interest could be handled by a separate forwarding strategy. This means, there isn't a single strategy having authority over a Data that satisfies multiple pending Interests.
In some strategy designs, it is desirable for the strategy to manipulate or otherwise control Data forwarding. For example, a strategy may want to add or remove congestion marks on outgoing Data packets (#3797), or delay Data forwarding (question).
However, since multiple strategies may have authority over a Data, if they are allowed to control Data forwarding, their decisions would conflict. Thus, current strategy API disallows such control.

This issue is to modify strategy API to give a strategy authority over Data forwarding.
It is dependent upon a NDN protocol change that allows a Data to satisfy only one Interest.


Related issues

Related to NFD - Feature #3333: Design Interest digestNew

Blocks NFD - Feature #4279: Self-learning strategyIn Progress2017-09-27

History

#1 Updated by Junxiao Shi about 1 year ago

strategy-data-trigger_20170209.pptx gives my initial design.

It is dependent upon a NDN protocol change that allows a Data to satisfy only one Interests.

One option for such protocol change is #3333. However, I'm not yet claiming #3333 as a blocking issue. A different protocol change that fulfills "one Data satisfies one Interest" condition is also acceptable.

Alternatively, this issue is already implementable for an incoming Data that happen to satisfy only one Interest, which I believe is true most of the time. Specifically, the incoming Data pipeline can count how many pending Interests are being satisfied. If this count equals one, the strategy is permitted control over Data forwarding; if this count is greater than one, the Data is forwarded according to the existing rules in forwarding pipelines and no strategy can manipulate it.

#2 Updated by Junxiao Shi about 1 year ago

#3 Updated by Junxiao Shi about 1 year ago

  • Blocks Feature #3797: Congestion Control: generic congestion marks added

#4 Updated by Davide Pesavento about 1 year ago

  • Description updated (diff)

#5 Updated by Klaus Schneider about 1 year ago

Junxiao Shi wrote:

It is dependent upon a NDN protocol change that allows a Data to satisfy only one Interests.

One option for such protocol change is #3333. However, I'm not yet claiming #3333 as a blocking issue. A different protocol change that fulfills "one Data satisfies one Interest" condition is also acceptable.

How about the following design? (copied from the email I sent you)

When a Data packet arrives:
1. Create a duplicate for each Interest it satisfies.
2. Then let each strategy handle its copy of the data packet as it wants.

#6 Updated by Junxiao Shi about 1 year ago

Reply to note-5:

In C-D-E topology, suppose E receives a Data /A/B from local app that satisfies pending Interest /A and /A/B. E would duplicate the Data as two copies to be processed by the effective strategy for each PIT entry. Normally, the strategy would send the Data to D. D would receive two copies of the Data.
When D receives the first copy of the Data, following the same rules, it would duplicate the Data again for each pending Interest and send two copies to C. When the second copy arrives at D, D would drop it as unsolicited. As a result, D could not make use of any congestion mark or lack thereof attached to the second Data copy from E.

The essence of #3333 is to explicit associate a Data copy with a pending Interest through a packet header named "Interest digest". Merely duplicating the Data for each PIT entry would not help.

#7 Updated by Klaus Schneider about 1 year ago

Okay, that makes sense.

I think we can go ahead and implement this feature for Data that satisfies only one Interest, as you described in note-1.

#8 Updated by Davide Pesavento about 1 year ago

In NFD, each pending Interest could be handled by a separate forwarding strategy.

How common is this scenario?

#9 Updated by Klaus Schneider about 1 year ago

Davide Pesavento wrote:

In NFD, each pending Interest could be handled by a separate forwarding strategy.

How common is this scenario?

Probably about as rare as a Data packet satisfying more than one pending Interest.

I'm fine with just ignoring that case, and implementing the new pipelines for the case of one Interest-one Data. Should work fine in >99% of all cases.

#10 Updated by Davide Pesavento about 1 year ago

But the problem here is really caused by having multiple strategies with "authority" over the same Data packet, not by the fact that one Data packet can satisfy multiple Interests. So why don't we ignore the case of multiple strategies, instead of that of satisfying multiple Interests?

#11 Updated by Klaus Schneider about 1 year ago

That should also work and includes a couple more cases.

So the logic would be:

if (satisfiesMultipleInterests && usesSeparateStrategies) {
  // old design: strategy can't manipulate Data
}
else {
  // new design: strategy *can* manipulate Data
}

#12 Updated by Klaus Schneider about 1 year ago

  • Priority changed from Normal to High

#13 Updated by Davide Pesavento about 1 year ago

  • Blocks deleted (Feature #3797: Congestion Control: generic congestion marks)

#14 Updated by Davide Pesavento about 1 year ago

  • Blocks Feature #1624: Design and Implement Congestion Control added

#15 Updated by Junxiao Shi 12 months ago

But the problem here is really caused by having multiple strategies with "authority" over the same Data packet, not by the fact that one Data packet can satisfy multiple Interests.

Correct.

So why don't we ignore the case of multiple strategies, instead of that of satisfying multiple Interests?

Incoming Data pipeline triggers strategy for each PIT entry. It would be more complicated to compare whether all PIT entries use the same strategy.

#16 Updated by Davide Pesavento 11 months ago

#17 Updated by Klaus Schneider 11 months ago

this issue is already implementable for an incoming Data that happen to satisfy only one Interest

I'm currently running some ndnSIM experiments, and found that only supporting this case (Data satisfying one Interest) is not good enough.

Example use case: I tried to send two simultaneous probing packets (same name, different nonce) which can get aggregated in one PIT entry. At the aggregating router, the returning Data should be updated differently for both outgoing data packets, based on the congestion level of each link's outgoing queue.

This requires that the strategy is given authority at least in the case when Data only satisfies one PIT entry (rather than just one Interest).

Junxiao Shi wrote:

So why don't we ignore the case of multiple strategies, instead of that of satisfying multiple Interests?

Incoming Data pipeline triggers strategy for each PIT entry. It would be more complicated to compare whether all PIT entries use the same strategy.

So how about we ignore the case of Data satisfying multiple PIT entries?

#18 Updated by Junxiao Shi 11 months ago

this issue is already implementable for an incoming Data that happen to satisfy only one Interest

This requires that the strategy is given authority at least in the case when Data only satisfies one PIT entry (rather than just one Interest).

Yes, "one Interest" meant "one PIT entry", regardless of how many downstreams it has.

So how about we ignore the case of Data satisfying multiple PIT entries?

Yes. Once incoming Data pipeline finds multiple PIT matches, the strategy cannot control Data forwarding, but can observe Data arrival as the case today.

#19 Updated by Klaus Schneider 11 months ago

Okay, that design sounds good to me.

#20 Updated by Klaus Schneider 10 months ago

  • Blocks deleted (Feature #1624: Design and Implement Congestion Control)

#21 Updated by Klaus Schneider 10 months ago

  • Assignee deleted (Klaus Schneider)

#22 Updated by Teng Liang 9 months ago

Junxiao Shi wrote:

Yes. Once incoming Data pipeline finds multiple PIT matches, the strategy cannot control Data forwarding, but can observe Data arrival as the case today.

How does the current code prevent fw manipulating Data?

#23 Updated by Junxiao Shi 9 months ago

How does the current code prevent fw manipulating Data?

“Prevent” is the wrong word. Current API never allowed strategy to manipulate Data. Any attempt to manipulate Data results in undefined behavior (i.e. not guaranteed/tested to work).

#24 Updated by Teng Liang 8 months ago

According to the discussions, people seem to aggree with adding an API to manipulate Data when only one pit entry is matched. How about the following logic:

for (pitEntry : pitMatches) {

  if (pitMatches.size() == 1) {
     dispatchToStrategy(strategy.canManipulateDataBeforeSatisfyInterest);
  }

  dispatchToStrategy(strategy.beforeSatisfyInterest);
}

But within the method Forwarder::onIncomingData(Face& inFace, const Data& data), data cannot be manipulated (such as adding a prefix announcement tag). How to support it with minimum changes to exiting code?

#25 Updated by Junxiao Shi 8 months ago

if (pitMatches.size() == 1) {
dispatchToStrategy(strategy.canManipulateDataBeforeSatisfyInterest);
}

This is insufficient. The strategy should be given full control over the Data, including dropping or delaying the Data. Thus, forwarding should pass the Data to the strategy via a trigger and stop further processing. The strategy can modify/delay Data as it wants, and then invoke actions (methods on Strategy base class) to send the Data out.

#26 Updated by Teng Liang 8 months ago

Junxiao Shi wrote:

This is insufficient. The strategy should be given full control over the Data, including dropping or delaying the Data. Thus, forwarding should pass the Data to the strategy via a trigger and stop further processing. The strategy can modify/delay Data as it wants, and then invoke actions (methods on Strategy base class) to send the Data out.

For self-learning, it only needs to attach a prefix announcement tag to Data. What well happen if self-learning does it in the beforeSatisfyInterest method.

#27 Updated by Junxiao Shi 8 months ago

Stop saying "for self-learning". This is a generic feature asked by many on nfd-dev and ndnSIM community. The API needs to be general.

#28 Updated by Teng Liang 8 months ago

Junxiao Shi wrote:

Stop saying "for self-learning". This is a generic feature asked by many on nfd-dev and ndnSIM community. The API needs to be general.

What will happen if forwarding strategy attaches a tag to Data in the beforeSatisfyInterest method?

#29 Updated by Junxiao Shi 8 months ago

What will happen if forwarding strategy attaches a tag to Data in the beforeSatisfyInterest method?

It triggers undefined behavior. beforeSatisfyInterest API does not allow modifying Data in any way, including changing tags.

#30 Updated by Teng Liang 8 months ago

  • Assignee set to Teng Liang

How do people like Junxiao's design (strategy-data-trigger_20170209.pptx)?

The two new trigger "after hit CS trigger" and "after receive Data trigger" to replace "before satisfy Interest trigger" looks good to me, except that I am not convinced to implement the feature of "extend PIT entry lifetime".

A compatability question is how to seamlessly change the APIs in all forwarding strategy, e.g., to submit the changes in all strategy along with this feature?

#31 Updated by Junxiao Shi 8 months ago

How do people like strategy-data-trigger_20170209.pptx?

I like it. It supports more features and is less confusing.

I am not convinced to implement the feature of "extend PIT entry lifetime".

The design eliminates the global "straggler timer" that is a major source of confusion in strategy API. Strategies that need this feature would have to invoke "extend PIT entry lifetime" action to retain the PIT entry.
NCC and best-route do not depend on this. I'm unsure about ASF. Self learning needs it to learn non fastest paths as backups.

A compatability question is how to seamlessly change the APIs in all forwarding strategy

  1. Add new APIs. Mark old APIs as deprecated.
  2. Report a bug for each strategy that invokes old API. Assign them to author of the strategy.
  3. Reject all new or updated strategies that invokes API in code reviews.
  4. Delete old APIs and also the straggler timer when step2 bugs are closed.

to submit the changes in all strategy along with this feature?

Don't try this. It'll produce a unusually large commit and you'll have trouble to find people to review it.

#32 Updated by Teng Liang 8 months ago

How about making two separate submits, the first one to simplify the APIs about the PIT entry lifetime, and the second one to change the APIs related Data pipelines?

#33 Updated by Teng Liang 8 months ago

Junxiao, in your slides:

Unsatisfy timer expiration invokes "before remove pending Interest" trigger.
* The intention was for strategy to clean up states, but all existing strategies use ScopedEventId or similar for cleanups and do not use this trigger.

Do you suggest that remove this trigger? Since the unsatify and stragger timers point to different triggers, merging them may end with an API that has complicated parameters. If the trigger for unsatify timer is removed, it needs a different design.

#34 Updated by Junxiao Shi 8 months ago

How about making two separate submits, the first one to simplify the APIs about the PIT entry lifetime, and the second one to change the APIs related Data pipelines?

If you need to ask about commit arrangement, be very specific about what each commit contains.

Since the unsatify and stragger timers point to different triggers, merging them may end with an API that has complicated parameters. If the trigger for unsatify timer is removed, it needs a different design.

Both unsatisfy timer (last in-record expiration time) and straggler timer (100ms after last out-record is rejected or strategy rejects Interest) are gone.
The new expiration timer differs from both old timers. See page11-13 of strategy-data-trigger_20170209.pptx for its exact semantics.

#35 Updated by Teng Liang 8 months ago

Junxiao Shi wrote:

How about making two separate submits, the first one to simplify the APIs about the PIT entry lifetime, and the second one to change the APIs related Data pipelines?

If you need to ask about commit arrangement, be very specific about what each commit contains.

Since the unsatify and stragger timers point to different triggers, merging them may end with an API that has complicated parameters. If the trigger for unsatify timer is removed, it needs a different design.

Both unsatisfy timer (last in-record expiration time) and straggler timer (100ms after last out-record is rejected or strategy rejects Interest) are gone.
The new expiration timer differs from both old timers. See page11-13 of strategy-data-trigger_20170209.pptx for its exact semantics.

I agree with most of your design, but I have a different implementation plan that includes two commits:

  1. changes that do not break any strategies, including simplifying PIT entry timers (but not to change the semantics of straggler timer), and removing unused trigger (reject interest trigger is used by 6 strategies, so not included).

  2. the rest changes mentioned in strategy-data-trigger_20170209.pptx, including add triggers to handle Data pipelines, removing Interest rejection trigger, and changing timer settings, need a broader discussion before being implemented, because this commit breaks existing strategies.

#36 Updated by Davide Pesavento 8 months ago

Teng Liang wrote:

the rest changes mentioned in strategy-data-trigger_20170209.pptx, including add triggers to handle Data pipelines, removing Interest rejection trigger, and changing timer settings, need a broader discussion before being implemented, because this commit breaks existing strategies.

You cannot break existing strategies.

#37 Updated by Teng Liang 8 months ago

Davide Pesavento wrote:

Teng Liang wrote:

the rest changes mentioned in strategy-data-trigger_20170209.pptx, including add triggers to handle Data pipelines, removing Interest rejection trigger, and changing timer settings, need a broader discussion before being implemented, because this commit breaks existing strategies.

You cannot break existing strategies.

That's what I wanted to avoid. These changes need to be discussed first. Even after we agree to implement them, we have make sure they are backward compatible.

#38 Updated by Davide Pesavento 8 months ago

Teng Liang wrote:

Even after we agree to implement them, we have make sure they are backward compatible.

They don't need to be backward compatible. You can change things in a backward incompatible manner, as long as you update all existing strategies so that they still work after your changes.

#39 Updated by Teng Liang 8 months ago

  • File Interest-workflow added
  • File Data-workflow added

Here is my implementation plan:

  1. In the proposed Interest workflow, the OnContentStoreHit pipeline stage will point to the new afterHitCs trigger in strategy, instead of the existing onOutgoingData pipeline stage.
  2. In the proposed Data workflow, the afterReceiveData trigger is added to strategy to handle Data when only one PIT entry is found.
  3. Use a single expiration timer to replace unsatisfy timer and straggler timer, and remove onInterestUnsatisfied and strategy::beforeExpirePendingInterest (which have not been used in any strategy).

These changes basically follow the descriptions in Junxiao's design (strategy-data-trigger_20170209.pptx), except that I keep Interest rejection API (onInterestReject & strategy::rejectPendingInterest) for backward compatibility and its user-friendly design. These changes should not require any change from existing strategies. The uncertain part to me is that whether the new timer would break any forwarding strategy, and whether a timer extension API is needed (if needed, how to design it).

#40 Updated by Teng Liang 8 months ago

  • File Data-workflow.jpg added
  • File Interest-workflow.jpg added

#41 Updated by Teng Liang 8 months ago

  • File deleted (Interest-workflow)

#42 Updated by Teng Liang 8 months ago

  • File deleted (Data-workflow)

#43 Updated by Teng Liang 8 months ago

Teng Liang wrote:

The uncertain part to me is that whether the new timer would break any forwarding strategy.

I tried the new expiration timer settings for PIT entry in different pipeline stages (https://gerrit.named-data.net/#/c/4556/5..6/daemon/fw/forwarder.cpp), and it passed all daemon test cases. I assume it works.

#44 Updated by Junxiao Shi 8 months ago

https://gerrit.named-data.net/4556 patchset8 is trying to do two things together:

  1. Eliminate "before expire pending Interest" strategy trigger.
  2. Merge "unsatisfy timer" and "straggler timer" into one field without changing its semantics.

The first change makes sense, because "before expire pending Interest" strategy trigger is too limited and it has not been used.
The second change, however, does not show clear benefit, but increases confusion and uncertainty on correctness.
I suggest reverting the second change and only keeping the first change.


Also, to modify anything in forwarding pipelines, please provide an update to forwarding-pipelines_*.pptx file. You can find this file in https://gitlab.com/named-data/TR-NDN-0021-NFD-dev-guide repository under slides directory. You may upload updated version of this file, renamed as the current date, to this Redmine issue. After uploading a version, never delete it, but upload additional versions with new filenames (you may append a letter after the date in case there are multiple versions on the same day). Once the changes are agreed upon, you can commit the last file into the repository replacing the old one, and modify the text and figures in devguide itself.

#45 Updated by Teng Liang 8 months ago

Junxiao Shi wrote:

https://gerrit.named-data.net/4556 patchset8 is trying to do two things together:

  1. Eliminate "before expire pending Interest" strategy trigger.
  2. Merge "unsatisfy timer" and "straggler timer" into one field without changing its semantics.

The first change makes sense, because "before expire pending Interest" strategy trigger is too limited and it has not been used.
The second change, however, does not show clear benefit, but increases confusion and uncertainty on correctness.
I suggest reverting the second change and only keeping the first change.

I agree to change the semantics for the new timer (not keeping the old timer semantics), and I think the first change is part of the second change. So these changes can go together, but I also agree to separate them, if people think it is necessary.

Also, to modify anything in forwarding pipelines, please provide an update to forwarding-pipelines_*.pptx file. You can find this file in https://gitlab.com/named-data/TR-NDN-0021-NFD-dev-guide repository under slides directory. You may upload updated version of this file, renamed as the current date, to this Redmine issue. After uploading a version, never delete it, but upload additional versions with new filenames (you may append a letter after the date in case there are multiple versions on the same day). Once the changes are agreed upon, you can commit the last file into the repository replacing the old one, and modify the text and figures in devguide itself.

Thanks. Do you mean that to keep a local repo until the final commit?

#46 Updated by Junxiao Shi 8 months ago

Thanks. Do you mean that to keep a local repo until the final commit?

No, you don't need a local repository. You just need to update the slides, which is a single file. Updated slides should be uploaded to Redmine only, and uploaded versions should never be deleted.

#47 Updated by Teng Liang 8 months ago

Junxiao Shi wrote:

https://gerrit.named-data.net/4556 patchset8 is trying to do two things together:

  1. Eliminate "before expire pending Interest" strategy trigger.
  2. Merge "unsatisfy timer" and "straggler timer" into one field without changing its semantics.

The first change makes sense, because "before expire pending Interest" strategy trigger is too limited and it has not been used.
The second change, however, does not show clear benefit, but increases confusion and uncertainty on correctness.
I suggest reverting the second change and only keeping the first change.

Actually pathset8 uses the new timer semantics described in strategy-data-trigger_20170209.pptx. With the new timer semantics, do you agree that these two changes go into one commit?

#48 Updated by Junxiao Shi 8 months ago

With the new timer semantics, do you agree that these two changes go into one commit?

They need to be separate as these are independent changes.

Actually pathset8 uses the new timer semantics described in strategy-data-trigger_20170209.pptx.

As I already said, “extend PIT entry lifetime” strategy action is necessary. A strategy action must appear as a protected method on Strategy base class, because a strategy does not have access to Forwarder instance.

#49 Updated by Davide Pesavento 8 months ago

Teng Liang wrote:

do you agree that these two changes go into one commit?

No, don't ever do that, unless the changes are really trivial and self-contained.

#50 Updated by Teng Liang 8 months ago

Junxiao Shi wrote:

With the new timer semantics, do you agree that these two changes go into one commit?

They need to be separate as these are independent changes.

Well, unsatisfy timer points to onInterestUnsatisfied, so they are somewhat dependent. However, I agree to make two commits.

#52 Updated by Junxiao Shi 8 months ago

forwarding-pipelines_20180226.pptx has the following errors or inconsistencies with the design given in strategy-data-trigger_20170209. Note that not all errors are caused by this updated, but it is required to fix all errors before proceeding.

  • ContentStore hit pipeline did not specify the duration of PIT expiry timer. The duration should be zero. Unless the strategy extends PIT entry lifetime, the PIT entry would be finalized as soon as the strategy trigger returns.
  • In ContentStore hit pipeline "dispatch matched Data in CS to strategy" step, it is incorrect to specify "which strategy should process this Data". Effective strategy solely depends on Interest. It is the Interest that determines which strategy would be triggered.
  • ContentStore miss pipeline did not specify the duration of PIT expiry timer. The PIT expiry timer should expire when the last PIT in-record expires. Unless the strategy extends or shortens PIT entry lifetime, the PIT entry would be finalized when the last PIT in-record expires, i.e. when no downstream is still expecting a response.
  • There shouldn't be an Interest reject pipeline. To reject an Interest, the strategy should invoke change PIT entry lifetime strategy action and set the PIT entry lifetime to be zero. Consequently, the PIT entry would be finalized as soon as the strategy trigger returns. Note that although Interest reject pipeline and reject pending Interest strategy action no longer exist, you may still retain Strategy::rejectPendingInterest as a helper function that invokes change PIT entry lifetime strategy action.
  • incoming Data pipeline did not specify the duration of PIT expiry timer. The duration should be zero. Unless the strategy extends PIT entry lifetime, the PIT entry would be finalized as soon as the after receive Data strategy trigger returns.
  • A send Data strategy action is needed for use in after receive Data. Its parameters are similar to send Nack strategy action. A helper function Strategy::sendAllData may be provided similar to Strategy::sendNacks.
  • In several pages, strategy trigger is being inconsistently referred to as "After CS hit", "invoke incoming Interest callback", etc. They should use the form "trigger strategy: after CS hit", "trigger strategy: incoming Interest", etc, and color-coded in "strategy" category (hint: use PowerPoint "quick style" for color coding; don't edit colors directly).
  • incoming Interest pipeline and outgoing Nack pipeline determine whether to send Nack with "is multi-access face" condition, but incoming Nack pipeline determine whether to accept Nack with "is point-to-point face" condition. They should use the same condition.

As a reminder, when you update forwarding-pipelines_*.pptx, never delete an older version. This is to ensure comments like this can be interpreted against a certain version. If you mistakenly delete an older version, you'll be asked to re-upload it before any further review would be given.

#53 Updated by Davide Pesavento 8 months ago

Junxiao Shi wrote:

  • A send Data strategy action is needed for use in after receive Data. Its parameters are similar to send Nack strategy action. A helper function Strategy::sendAllData may be provided similar to Strategy::sendNacks.

The meaning of sendAllData is unclear/misleading. sendDataToAll would be much more intuitive. You may consider renaming the existing sendNacks to sendNackToAll for consistency.

#54 Updated by Teng Liang 8 months ago

Junxiao Shi wrote:

  • ContentStore hit pipeline did not specify the duration of PIT expiry timer. The duration should be zero. Unless the strategy extends PIT entry lifetime, the PIT entry would be finalized as soon as the strategy trigger returns.
  • incoming Data pipeline did not specify the duration of PIT expiry timer. The duration should be zero. Unless the strategy extends PIT entry lifetime, the PIT entry would be finalized as soon as the after receive Data strategy trigger returns.

In the previous slides, I put the all PIT expiry settings together in page 20. In the new update, I put them separately in each detailed slide, to be clearer.

  • ContentStore miss pipeline did not specify the duration of PIT expiry timer. The PIT expiry timer should expire when the last PIT in-record expires. Unless the strategy extends or shortens PIT entry lifetime, the PIT entry would be finalized when the last PIT in-record expires, i.e. when no downstream is still expecting a response.

The last PIT in-record is the record for the incoming Interest. Why not directly use the Interest lifetime + now as the expiry timer?

  • There shouldn't be an Interest reject pipeline. To reject an Interest, the strategy should invoke change PIT entry lifetime strategy action and set the PIT entry lifetime to be zero. Consequently, the PIT entry would be finalized as soon as the strategy trigger returns. Note that although Interest reject pipeline and reject pending Interest strategy action no longer exist, you may still retain Strategy::rejectPendingInterest as a helper function that invokes change PIT entry lifetime strategy action.

I agree to remove Interest reject pipeline stage, but to keep Strategy::rejectPendingInterest for backward compatibility and its user-friendly design.

A helper function Strategy::sendAllData may be provided similar to Strategy::sendNacks.

The names in Note-53 are taken for this one.

  • incoming Interest pipeline and outgoing Nack pipeline determine whether to send Nack with "is multi-access face" condition, but incoming Nack pipeline determine whether to accept Nack with "is point-to-point face" condition. They should use the same condition.

I changed Interest loop and outgoing Nack pipeline stages for this one.

I agree with other points and made changes to the updated slides (*_20180227.pptx).

#55 Updated by Klaus Schneider 8 months ago

Eliminate "before expire pending Interest" strategy trigger.

The first change makes sense, because "before expire pending Interest" strategy trigger is too limited and it has not been used.

A bit late since this has already been merged, but:

  • How do you know this features hasn't been used be researchers? There's a lot more strategies used in published research than the ones shipping with NFD.

  • Why is it "too limited"? How do you plan to avoid these limitations?

#56 Updated by Junxiao Shi 8 months ago

Eliminate "before expire pending Interest" strategy trigger.

"before expire pending Interest" strategy trigger is too limited and it has not been used.

  • How do you know this features hasn't been used be researchers? There's a lot more strategies used in published research than the ones shipping with NFD.

NFD never guarantees backwards compatibility. Third parties either stay with old version, or switch to new API.

  • Why is it "too limited"? How do you plan to avoid these limitations?

Strategy does not usually care about when the PIT entry disappears, but sometimes needs a timed trigger earlier than that (e.g. RTO).
For the former, strategy can know when PIT entry disappears by adding a StrategyInfo on the PIT entry and waiting for its destructor invocation. “before expire pending Interest” trigger was designed for cleaning up StrategyInfo but it’s obviously better to use the destructor.
For the latter, strategy can use the scheduler.

#57 Updated by Junxiao Shi 8 months ago

  • ContentStore miss pipeline did not specify the duration of PIT expiry timer. The PIT expiry timer should expire when the last PIT in-record expires. Unless the strategy extends or shortens PIT entry lifetime, the PIT entry would be finalized when the last PIT in-record expires, i.e. when no downstream is still expecting a response.

The last PIT in-record is the record for the incoming Interest. Why not directly use the Interest lifetime + now as the expiry timer?

The last PIT in-record to expire is not necessarily the last incoming Interest. Consider the following:

  1. At time 0, an Interest arrives from downstream 1001 with lifetime 4000. Downstream 1001’s in-record is to expire at time 4000. PIT entry’s expiry timer shall expire at time 4000.
  2. At time 1000, an Interest arrives from downstream 1002 with lifetime 1500. Downstream 1002’s in-record is to expire at time 2500. PIT entry’s expiry timer shall expire at time 4000, not 2500.
  3. At time 2000, an Interest arrives from downstream 1001 with lifetime 200. Downstream 1001’s in-record is to expire at time 2200, shortened from the previous 4000. PIT entry’s expiry timer shall expire at time 2500.

#58 Updated by Davide Pesavento 8 months ago

  • Status changed from New to In Progress

#59 Updated by Junxiao Shi 7 months ago

Teng: If you have completed forwarding updates, please remember to write NFD devguide. I will not approve any strategy code changes until NFD devguide updates are uploaded.

#60 Updated by Teng Liang 7 months ago

forwarding-pipelines_20180403.pptx is added. Page 5 - Pipelines Overall Workflow and page 13 - Incoming Data pipeline are modified, to describe the design of code change.

#61 Updated by Junxiao Shi 7 months ago

forwarding-pipelines_20180403.pptx does not render correctly.

  • Ensure all text (except "Y" "N" on the arrow) are embedded in a shape instead of in a separate textbox. To embed text in a shape, select the shape then type over it. If text exceeds shape boundary, you may uncheck "Format Shape - Shape Options - Text Box - Wrap text in shape" option, and insert line breaks manually.
  • Ensure both ends of each arrow is anchored to a shape.
  • Width and height of most shapes should be multiple of 0.3in.

#62 Updated by Teng Liang 7 months ago

Junxiao Shi wrote:

forwarding-pipelines_20180403.pptx does not render correctly.

Which software and platform are you using? It works perfect on Mac using Microsoft PowerPoint and Ubuntu using LibreOffice Impress.

  • Ensure all text (except "Y" "N" on the arrow) are embedded in a shape instead of in a separate textbox. To embed text in a shape, select the shape then type over it. If text exceeds shape boundary, you may uncheck "Format Shape - Shape Options - Text Box - Wrap text in shape" option, and insert line breaks manually.
    You cannot type over a line on Mac's PowerPoint, and I see no problem of using a textbox and group it with the line.

  • Ensure both ends of each arrow is anchored to a shape.
    Although it does not really affect understanding, I changed the the arrow type in the latest update.

  • Width and height of most shapes should be multiple of 0.3in.
    Obviously, this will bad and not fit in a page. The current shape looks better to me.

#63 Updated by Teng Liang 7 months ago

Junxiao Shi wrote:

forwarding-pipelines_20180403.pptx does not render correctly.

Which software and platform are you using? It works perfect on Mac using Microsoft PowerPoint and Ubuntu using LibreOffice Impress.

  • Ensure all text (except "Y" "N" on the arrow) are embedded in a shape instead of in a separate textbox. To embed text in a shape, select the shape then type over it. If text exceeds shape boundary, you may uncheck "Format Shape - Shape Options - Text Box - Wrap text in shape" option, and insert line breaks manually.

You cannot type over a line on Mac's PowerPoint, and I see no problem of using a textbox and group it with the line.

  • Ensure both ends of each arrow is anchored to a shape.

Although it does not really affect understanding, I changed the the arrow type in the latest update.

  • Width and height of most shapes should be multiple of 0.3in.

Obviously, this will bad and not fit in a page. The current shape looks better to me.

#64 Updated by Junxiao Shi 7 months ago

My office uses Microsoft Office 365 in Windows 10. You can find the same in University Libraries.
Please fix all problems, or I won't review.

#65 Updated by Teng Liang 7 months ago

  • File Junxiao_s slide.png added
  • File Teng_s slide.png added

Junxiao Shi wrote:

My office uses Microsoft Office 365 in Windows 10. You can find the same in University Libraries.
Please fix all problems, or I won't review.

Can you see the slide page the same as what I see (Teng's slide.png)? In addition, smaller boxes make them more clear.

#66 Updated by Teng Liang 7 months ago

Junxiao Shi wrote:

forwarding-pipelines_20180403.pptx does not render correctly.

To solve the render problem on different platforms, we should use .pdf in future. I've converted the latest version into pdf.

#67 Updated by Junxiao Shi 7 months ago

To solve the render problem on different platforms, we should use .pdf in future.

It's insufficient. How can you guarantee the next editor would be able to have a correctly rendered PPTX before she updates the pipeline design?

You cannot type over a line on Mac's PowerPoint, and I see no problem of using a textbox and group it with the line.

The pipelines document shall be updated on Windows only.

  • Ensure both ends of each arrow is anchored to a shape.

Although it does not really affect understanding, I changed the the arrow type in the latest update.

It ensures the arrows stay in place when moving other elements.

  • Width and height of most shapes should be multiple of 0.3in.

Obviously, this will bad and not fit in a page. The current shape looks better to me.

This requirement forces you to rethink the organization of each diagram and ensures it is understandable. If a diagram becomes too large, split it into multiple diagrams.

#68 Updated by Eric Newberry 7 months ago

Junxiao Shi wrote:

You cannot type over a line on Mac's PowerPoint, and I see no problem of using a textbox and group it with the line.

The pipelines document shall be updated on Windows only.

Junxiao, you cannot force people to use Windows for a task since it's not free.

#69 Updated by Junxiao Shi 7 months ago

you cannot force people to use Windows for a task since it's not free.

See note-64 first line second sentence.

Apply the same logic: you cannot force people to support Odroid and macOS because I cannot afford these.

#70 Updated by Davide Pesavento 7 months ago

Junxiao Shi wrote:

you cannot force people to use Windows for a task since it's not free.

See note-64 first line second sentence.

Apply the same logic: you cannot force people to support Odroid and macOS because I cannot afford these.

It is not the same logic. The NDN project provides hardware to run CI on those platforms, you don't have to own those machines yourself.

Also, this is completely off-topic.

#71 Updated by Teng Liang 7 months ago

Junxiao Shi wrote:

  • Width and height of most shapes should be multiple of 0.3in.

Obviously, this will bad and not fit in a page. The current shape looks better to me.

This requirement forces you to rethink the organization of each diagram and ensures it is understandable. If a diagram becomes too large, split it into multiple diagrams.

I re-organized it as Teng's slide.png. Can you understand the content?

One change is that I made some shapes smaller and put the text under the shapes, in order to make it clearer. Is it clearer than your previous slides? Why are we wasting time on how to make slides style satisfied upon your standards?

#72 Updated by Teng Liang 6 months ago

@Junxiao, can you take a look on the latest version of design?

#73 Updated by Teng Liang 6 months ago

In forwarding-pipelines_20180409 files, on page 13 - Incoming Data pipeline, add "clear PIT out records" building block to the branch when only one PIT is matched.

#74 Updated by Teng Liang 6 months ago

  • File deleted (Data-workflow.jpg)
  • File deleted (Interest-workflow.jpg)

#75 Updated by Teng Liang 6 months ago

The NFD developer guide is updated accordingly (see commit 99302934a57448be268def81abf4b8619462d1f3). Please take a look. Thanks.

#77 Updated by Davide Pesavento 6 months ago

  • Status changed from In Progress to Feedback
  • % Done changed from 0 to 50

Is this issue complete apart from documentation?

#78 Updated by Teng Liang 6 months ago

Davide Pesavento wrote:

Is this issue complete apart from documentation?

Yes, I think so. It is better if Junxiao confirms it.

#79 Updated by Junxiao Shi 6 months ago

  • Status changed from Feedback to Closed
  • % Done changed from 50 to 100

I trust that NFD devguide has been updated, but I won't review it.

Also available in: Atom PDF