NFD » ndn-tools

Is there an obvious way to do the version discovery without those selectors?

Is there an obvious way to do the version discovery without those selectors?

You may use RDR protocol for version discovery (blocked by #4706 and #4707), or just require the caller to specify a version number.

just require the caller to specify a version number

We already have a fixed-version mode for version discovery that doesn't actually discover anything but requires the user to provide the exact version number. But that's not a valid solution. This task is about replacing the existing "iterative" method with something else that still performs a real version discovery, starting from a name without any version number.

This task is about replacing the existing "iterative" method with something else that still performs a real version discovery, starting from a name without any version number.

Apparently, right now we look at the very first 3 components (as both MaxSuffixComponents and MinSuffixComponents are set to 3) of the name prefix and use the right-most child to find the version number.

First of all, please let me know what the problem is with this solution??
Second, what do you mean with "starting from a name without any version number"??

Chavoosh Ghasemi wrote:

First of all, please let me know what the problem is with this solution??

MinSuffixComponents, MaxSuffixComponents, ChildSelector are deprecated and no longer exist in packet format v0.3

Second, what do you mean with "starting from a name without any version number"??

Meaning that the user provides a name prefix that doesn't already contain a version number component.

We can set the name of the very first discovery Interest as below:

<prefix>/VDISCOVERY

Upon receiving this Interest by the Producer it will reply by a Data packet that its name includes the version number (probably with no content).
Does this solution make sense to you?

Don't reinvent the wheel. See #4396 and #4706.

Reply to note-8: This is exactly same idea as the RDR protocol (see note-3). Therefore, please work on #4706 to define the protocol, and then implement the protocol under this task.

Cool, I just read RDR paper and as you mentioned it is exactly as same as my idea.
Anyways, to implement RDR, issues #4706, #4707, and #4577 should be solved. That sounds a lot of work! Any plan that how to manage them?

P.S. I am just wondering why RDR paper is out but its code is not.

According to my meeting with Beichuan, this task can be divided into 2 different steps:
1- Making the FixedVersionDiscovery the default discovery version, so applications can work with packet format v0.3 and disabling IterativeVersionDiscovery
2- Finding a solution for real version discovery (e.g. RDR)

I am going to start with the first step and push the changes. If you think we need a separate redmine issue for this one, let me know.

Chavoosh Ghasemi wrote:

According to my meeting with Beichuan, this task can be divided into 2 different steps:
1- Making the FixedVersionDiscovery the default discovery version, so applications can work with packet format v0.3 and disabling IterativeVersionDiscovery
2- Finding a solution for real version discovery (e.g. RDR)

I am going to start with the first step and push the changes. If you think we need a separate redmine issue for this one, let me know.

About #2: each consumer can do discovery by sending out a discovery interest with name //metadata (with the mustBeFresh and MustBePrefix fields enabled). When the producer receives this Interest, it should send back the version of the latest file in the content of the metadata packet with name //metadata/. The name component is a monotonically increasing integer (starting from 0) that is appended to each metadata packet generated by the producer, so that it has a unique name. The freshness period of the metadata packet should be relatively small (e.g., 10ms should be a good starting point).

After the discovery phase, the consumer can start fetching the segments of the file.

I would suggest to get rid of "modular" version discovery altogether and use the following logic instead: if the user-provided name contains a version number, use that one and start fetching immediately; otherwise use RDR to obtain the latest version and then fetch the discovered version.

I agree with this solution because it reduces complexity.

I would suggest to get rid of "modular" version discovery altogether and use the following logic instead: if the user-provided name contains a version number, use that one and start fetching immediately; otherwise use RDR to obtain the latest version and then fetch the discovered version.

I agree with this solution because it reduces complexity.

How does this help improve the content retrieval process?

20190213 NFD call recognizes that the API of PipelineInterests::run cannot effectively support RDR version discovery method.

The existing iterative version discovery method would retrieve a data segment during version discovery.
Under this assumption, PipelineInterests::run method accepts this data segment: its name is used to derive versioned prefix, and its payload is stored so that the same segment does not need to be re-fetched.

RDR verson discovery method can only learn the versioned prefix from the metadata packet, but does not have a data segment.
To support RDR, PipelineInterests class should be generalized: PipelineInterests::run should accept versioned prefix instead of a data segment.
This does mean that the segment found during iterative or fixed version discovery would be re-fetched, but it would be supplied by the local node CS right away and would not generate excess network traffic.

To pass the version number to pipeline (instead of data packet), the following steps can be considered:

• Change
  signal::Signal<DiscoverVersion, const Data&> onDiscoverySuccess in DiscoveryVersion class to
  signal::Signal<DiscoverVersion, const uint64_t> onDiscoverySuccess
  so it will accept version number and not data packet.

• Change
  run(const Data& data, DataCallback onData, FailureCallback onFailure) in PipelineInterests class to
  run(const uint64_t version, DataCallback onData, FailureCallback onFailure)

• Update Consumer class according to above changes

great, so we broke the default version discovery and the latest release of ndn-tools contains a broken-by-default chunks tool.
it would be a good idea to add a "unit" test that integrates consumer and producer with each other to ensure the pair of tools actually works together. If a unit test for this is infeasible, an integration test would still be better than nothing. But this doesn't have to be in this commit.

I thought it's in IntegrationTests but it's not. Yes, this should be structured as a unit test but it's out of scope of this issue.