Feature #2734
closedSegmentFetcher: asynchronous Data validation with Validator
100%
Description
SegmentFetcher::fetch uses a verifySegment callback which returns a Data validation result synchronously.
But verifying a Data packet may require network operations to fetch certificates.
SegmentFetcher
should accept a Validator
instance, and use its asynchronous validation API.
SegmentFetcher is a simple procedure that fetches one segment at a time.
To keep it simple, when validation is in progress, the next segment need not be fetched.
Updated by Anonymous over 9 years ago
- Blocks Task #2543: Port SegmentFetcher added
Updated by Junxiao Shi over 9 years ago
- Tracker changed from Feature to Bug
- Subject changed from Should SegmentFetcher.VerifySegment be async? to SegmentFetcher::VerifySegment should be asynchronous
- Description updated (diff)
- Category set to Utils
- Estimated time set to 3.00 h
Updated by Junxiao Shi over 9 years ago
Updated by Alex Afanasyev over 9 years ago
I would pose this issue in a slightly different terms. Why don't we plug in Validator (require Validator instance to be passed) into the fetcher, instead of callbacks?
Updated by Lan Wang over 9 years ago
We are using SegmentFetcher to implement an application that retrieves NLSR status dataset. This feature is very important for our implementation.
Updated by Alex Afanasyev over 9 years ago
- Target version set to v0.4
I would not block on #1872. If #1872 is implemented after this issue, then it would simply update what is necessary in SegmentFetcher
code and dependencies.
I would say that we should prioritize this issue, as it is important and relatively straightforward to implement. Fetching is already an async process and it was a mistake for VerifySegment
by synchronous.
Updated by Junxiao Shi over 9 years ago
- Subject changed from SegmentFetcher::VerifySegment should be asynchronous to SegmentFetcher should allow asynchronous Data validation
- Description updated (diff)
20150810 conference call approves the idea in note-4.
Updated by Junxiao Shi over 9 years ago
- Blocks Task #2965: LSAs should be segmented before they are published added
Updated by Junxiao Shi over 9 years ago
- Estimated time changed from 3.00 h to 5.00 h
I'm increasing estimated time because using Validator
needs more work in changing unit testing.
Updated by Junxiao Shi over 9 years ago
- Tracker changed from Bug to Feature
- Subject changed from SegmentFetcher should allow asynchronous Data validation to SegmentFetcher: asynchronous Data validation with Validator
- Start date deleted (
04/07/2015)
Updated by Junxiao Shi about 9 years ago
- Status changed from New to Code review
http://gerrit.named-data.net/2418
commit:a612b66d6573e5b5805ee74094aa784de82d287f changes SegmentFetcher
public API as:
static void fetch(Face& face, const Interest& baseInterest, Validator& validator, const CompleteCallback& completeCallback, const ErrorCallback& errorCallback);
One question was:
Since the changes to other projects require maintaining a ValidatorNull, should the SegmentFetcher class instead provide a static ValidatorNull that users can pass to fetch()?
While I won't comment specifically on ValidatorNull
, this is a general problem: the caller is forced to maintain a Validator
instance.
One solution is: add another fetch
overload that accepts shared_ptr<Validator>
in place of Validator&
.
Therefore, if the caller is unwilling to maintain the Validator
instance, it could use this overload and let the SegmentFetcher
keep the Validator
alive.
Updated by Alex Afanasyev about 9 years ago
I'm not against adding an overload, but what if we simply change fetch
signature to accept shared_ptr<Valiator>
? Would there be any negative consequences?
Updated by Junxiao Shi about 9 years ago
Answer to note-13:
Only accepting shared_ptr<Validator>
forces the caller to create Validator
with make_shared
, which is undesirable.
Many classes have been accepting Validator&
as a constructor parameter, and .shared_from_this()
imposes additional requirements on the caller.
Updated by Muktadir Chowdhury about 9 years ago
There are four kinds of nack.
if (nack == NONE) {
reexpress the interest;
}
else if (nack == CONGESTION) {
reexpress the interest after waiting a while;
}
else if (nack == DUPLICATE) {
wait for timeout/data;
}
else if (nack == NO_ROUTE) {
error;
}
Updated by Junxiao Shi about 9 years ago
note-15 design is incomplete:
- "NONE" is not a NackReason. It indicates a Nack with no reason.
- Application SHOULD treat unrecognized NackReason as a Nack with no reason.
- After DUPLICATE, neither timeout nor Data would come back.
Recommendation:
switch (nack.getReason()) {
case DUPLICATE:
re-express without delay, up to 3 times;
break;
case CONGESTION:
re-express with exponential back-off, up to 3 times;
break;
default:
fail;
}
Updated by Junxiao Shi about 9 years ago
How do we coordinate this commit to be merged?
This is a breaking change, so it requires a 5-day notice to ndn-lib mailing list.
Also, prepare patches for NFD, NLSR, and other projects.
ndn-cxx-breaks can help with this process.
Updated by Junxiao Shi about 9 years ago
- Blocks Task #3334: Deprecate Face::expressInterest non-Nack overloads added
Updated by Muktadir Chowdhury about 9 years ago
- Link of the result of the build of all NDN projects (except ChronoChat): https://travis-ci.org/yoursunny/ndn-cxx-breaks/builds/90789119
- Link to the notice in ndn-lib archive: http://www.lists.cs.ucla.edu/pipermail/ndn-lib/2015-November/000314.html
- Link to affected projects changes in Gerrit:
Updated by Vince Lehman about 9 years ago
I can confirm that ChronoChat (http://gerrit.named-data.net/#/c/2584/2) compiles with this ndn-cxx patch (http://gerrit.named-data.net/#/c/2418/26).
laptop:ndn-cxx vince$ grep BUILD_STRING /usr/local/include/ndn-cxx/version.hpp | tail -n 1
#define NDN_CXX_VERSION_BUILD_STRING "0.1.0-371-gf9f5dc1"
…
laptop:ChronoChat vince$ git rev-parse HEAD
5119f3d84a8abcd7c538f9216587a12ea65b48b9
…
laptop:ChronoChat vince$ ./waf
…
[73/75] Linking build/ChronoChat
[74/75] Compiling build/ChronoChat
[75/75] Creating build/ChronoChat.app/Contents/Info.plist
'build' finished successfully (44.015s)
Updated by Alex Afanasyev almost 9 years ago
- Status changed from Code review to Closed
- % Done changed from 0 to 100
Updated by Junxiao Shi over 8 years ago
- Status changed from Closed to Feedback
The test suite introduced with this feature is running slowly on my machine.
https://gerrit.named-data.net/2926 improves it.
Updated by Junxiao Shi over 8 years ago
Test duration comparison:
http://jenkins.named-data.net/job/ndn-cxx/3596/OS=Ubuntu-14.04-64bit/consoleText -> http://jenkins.named-data.net/job/ndn-cxx/3599/OS=Ubuntu-14.04-64bit/consoleText
test case | old duration | new duration |
---|---|---|
Timeout | 4373 | 2172 |
Triple | 2345 | 1370 |
MultipleSegmentFetching | 173322 | 57522 |
SegmentZero | 38669 | 1535 |