NFD

I will look into the passphrase issue separate from Tao's commit on Gerrit.

Junxiao Shi wrote:

You can use as many commits as necessary, but I won't merge any of them until all failures are fixed.

I disagree strongly with this. The test cases are all independent, so they can be fixed independently.

Eric Newberry wrote:

Junxiao Shi wrote:

You can use as many commits as necessary, but I won't merge any of them until all failures are fixed.

I disagree strongly with this. The test cases are all independent, so they can be fixed independently.

The test cases are all independent, so they can be fixed independently.

Then disable failing tests in Python. Vagrant verification procedure is seen as “automated”, same as how Jenkins executes NFD unit tests. If anything is broken, the verification cannot pass.

Junxiao Shi wrote:

The test cases are all independent, so they can be fixed independently.

Then disable failing tests in Python. Vagrant verification procedure is seen as “automated”, same as how Jenkins executes NFD unit tests. If anything is broken, the verification cannot pass.

They can pass if someone looks at the results and verifies that the test cases in question do in fact pass. We've had this issue in the past and have worked around it.

Junxiao Shi wrote:

Inside vagrant ssh A, it seems that NFD did not stop after a certain test case:

vagrant@vagrant:~$ pstree -p 2604
run_tests.py(2604)---run_tests.py(5641)---sudo(5644)---nfd(5646)-+-{nfd}(5648)
                                                                 `-{nfd}(5649)

After typing nfd-stop within the VM, the outer script proceeds.

This is probably related to #4410 and Ubuntu 16.04.

As of the latest patchset of the change to fix this issue (4450,17 rebased onto the current Git HEAD), the following test cases are failing:

• test_hub_discovery
• test_remote_register
• test_aggregation

The issue described in #4410 remains present as well.

In regards to test_aggregation, it is most likely due to "Peek Client 4" using the deprecated ChildSelector selector (test_interest_aggregation.py line 125), which will cause it to fail with a return code of 2 on systems with newer versions of NDN-tools.

test_aggregation is failing because it is attempting to supply an argument to ndnpeek that was removed recently (set rightmost ChildSelector). This seems to be a larger issue than simply fixing a failure due to changed tool syntax or output format. The test README lists this for the failing step:

• Start ndnpeek to request ndn:/test-agg/B ChildSelector=leftmost, and another instance to request ndn:/test-agg/B ChildSelector=rightmost. Wait until they terminate. Fail if either is unanswered.

Can this be tested using the new Interest elements, or should this test be removed?

Alexander Lane wrote:

In regards to test_aggregation, it is most likely due to "Peek Client 4" using the deprecated ChildSelector selector (test_interest_aggregation.py line 125), which will cause it to fail with a return code of 2 on systems with newer versions of NDN-tools.

Just saw this right as I was posting my note about the same thing. This test case needs to be redesigned to use the new Interest elements.

Also, the integration tests (unless told otherwise), always install the latest version of ndn-tools from git on the system, so this will always occur when running the integration tests with the default options, no matter the version of ndn-tools previously installed on the system.

Eric Newberry wrote:

• Start ndnpeek to request ndn:/test-agg/B ChildSelector=leftmost, and another instance to request ndn:/test-agg/B ChildSelector=rightmost. Wait until they terminate. Fail if either is unanswered.

Can this be tested using the new Interest elements, or should this test be removed?

Just delete this particular test. ChildSelector is deprecated.

Davide Pesavento wrote:

Eric Newberry wrote:

• Start ndnpeek to request ndn:/test-agg/B ChildSelector=leftmost, and another instance to request ndn:/test-agg/B ChildSelector=rightmost. Wait until they terminate. Fail if either is unanswered.

Can this be tested using the new Interest elements, or should this test be removed?

Just delete this particular test. ChildSelector is deprecated.

This particular test or the entire test_integration_aggregation test case?

test_aggregation is failing because it is attempting to supply an argument to ndnpeek that was removed recently (set rightmost ChildSelector). This seems to be a larger issue than simply fixing a failure due to changed tool syntax or output format. The test README lists this for the failing step:

• Start ndnpeek to request ndn:/test-agg/B ChildSelector=leftmost, and another instance to request ndn:/test-agg/B ChildSelector=rightmost. Wait until they terminate. Fail if either is unanswered.

The intention of this step is to express two Interests with same name that should not be aggregated by PIT.
This step shall be replaced with:

• Start ndnpeek to request ndn:/test-agg/B CanBePrefix=false, and another instance to request ndn:/test-agg/B CanBePrefix=true. Wait until they terminate. Fail if either is unanswered.

This replacement step can achieve the same goal as original test case.

Currently broken tests (on Docker):

• test_auto_prefix_propagate (test_remote_register)
• test_hub_discovery
• test_interest_loop (test_client_control)
• test_linkfail (test_multicast, test_ncc)
• test_localhost_scope (test_localhost_scope_in, test_localhost_scope_out)
• test_mobility_link
• test_multicast (test_eth_multicast)
• test_nack
• test_ndntraffic