Project

General

Profile

Actions

Task #1388

closed

nfdc and nfd-status test scenario

Added by Yi Huang about 10 years ago. Updated almost 10 years ago.

Status:
Closed
Priority:
Normal
Category:
Integration Tests
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
5.00 h

Description

Topology: A-B connected on same IPv4 subnet and same IPv6 subnet. Both are NDN nodes.

Script is invoked on host A. Script should control other hosts over ssh.

multi-host.conf has these variables defined: CTRL_B, IP4_A1, IP6_A1, IP4_B1, IP6_B1

nfdc create / add-nexthop / destroy test case

  1. Start NFD on host A and B.
    Unless otherwise noted, perform all steps below on host A.
  2. Execute nfd-status. Check the existence of udp4/udp6/ether multicast faces.
  3. Invoke nfdc create tcp://$IP4_B1. Pause 2 seconds for connection establishment.
  4. Execute nfd-status. Check the existence of new tcp4 face.
  5. Invoke nfdc create tcp://[$IP6_B1]. Pause 2 seconds for connection establishment.
  6. Execute nfd-status. Check the existence of new tcp6 face.
  7. Invoke nfdc create udp://$IP4_B1. Pause 2 seconds for connection establishment.
  8. Execute nfd-status. Check the existence of new udp4 face.
  9. Invoke nfdc create udp://[$IP6_B1]. Pause 2 seconds for connection establishment.
  10. Execute nfd-status. Check the existence of new udp6 face.
  11. Invoke nfdc to add a nexthop for ndn:/test-nfdc with cost 24 toward FaceId created in step 3.
  12. Invoke nfdc to add a nexthop for ndn:/test-nfdc with cost 26 toward FaceId created in step 5.
  13. Invoke nfdc to add a nexthop for ndn:/test-nfdc with cost 14 toward FaceId created in step 7.
  14. Invoke nfdc to add a nexthop for ndn:/test-nfdc with cost 16 toward FaceId created in step 9.
  15. Execute nfd-status. Check the existence of ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
  16. Invoke nfdc to choose BroadcastStrategy for ndn:/ namespace.
  17. Invoke ndn-tlv-ping to send 100 Interests under ndn:/test-nfdc prefix.
    Pause 50ms before sending each Interest.
  18. Execute nfd-status. Verify NOutInterests counters of faces created in step 3,5,7,9 are greater than 60.
  19. On host B, execute nfd-status. Check the existence of tcp4, tcp6, udp4, udp6 faces with host A's address. Verify NInInterests counters for those faces are greater than 60.
  20. Invoke nfdc to destroy FaceIds created in step 3,5.
  21. Execute nfd-status. Check the non-existence of tcp4 and tcp6 faces.

nfdc add-nexthop / remove-nexthop test case

  1. Start NFD on host A and B.
    Unless otherwise noted, perform all steps below on host A.
  2. Invoke nfdc add-nexthop -c 44 ndn:/test-nfdc udp4://$IP4_B1. Pause 2 seconds for connection establishment.
  3. Execute nfd-status. Check the existence of new tcp4 face. Check the existence of ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
  4. Invoke nfdc to remove the nexthop for FaceId seen in step 3.
  5. Execute nfd-status. Check the non-existence of ndn:/test-nfdc FIB entry.

nfdc register / unregister test case

  1. Start NFD on host A and B.
    Unless otherwise noted, perform all steps below on host A.
  2. Invoke nfdc register -c 55 ndn:/test-nfdc udp4://$IP4_B1. Pause 2 seconds for connection establishment.
  3. Execute nfd-status. Check the existence of new tcp4 face. Check the existence of ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
  4. Invoke nfdc to unregister the prefix ndn:/test-nfdc for FaceId seen in step 3.
  5. Execute nfd-status. Check the non-existence of ndn:/test-nfdc FIB entry.

Script can assume integrated test suite is cloned to all hosts, necessary programs are installed and accessible in $PATH, and current user can ssh to other hosts and has sudo privilege.


Files

result_c858-5.tgz (55.1 KB) result_c858-5.tgz Junxiao Shi, 05/29/2014 03:50 PM
result_c858-6.zip (55.2 KB) result_c858-6.zip Junxiao Shi, 05/29/2014 11:22 PM
result_c858-7.tgz (55.3 KB) result_c858-7.tgz Junxiao Shi, 06/10/2014 02:54 PM
result_c858-8.zip (60.1 KB) result_c858-8.zip Junxiao Shi, 06/12/2014 12:07 AM
Actions #1

Updated by Junxiao Shi about 10 years ago

  • Subject changed from Test nfdc to Test nfdc and nfd-status
  • Description updated (diff)
  • Start date deleted (03/22/2014)
  • Estimated time set to 5.00 h
Actions #2

Updated by Hila Ben Abraham about 10 years ago

  • Status changed from New to In Progress

I have few technical questions:

  1. What is the right path to store this script? I thought tools/tests might be better than tests/, but I'm open for your suggestions.
  2. I'm gonna go for bash or python unless you have any other recommendation.
  3. return value/ printout: Are we going to use it as part of a regression test? if so, is there any return value / printout I should design the script accordingly? or is it sufficient to print something like "FAIL" or "SUCCESS" for now?
  4. Should the script exit for the first failure or should we run it through the end and provide one output log?
  5. nfdc exits with a success or failure message, should I check this before trying nfd-status?
Actions #3

Updated by Junxiao Shi about 10 years ago

  1. Integrated tests are stored in a separate repository which should be created during Task #1220.
  2. I believe any language is fine.
  3. Return zero and print nothing on success. Return non-zero and print something to stdout on failure.
  4. Either way is fine.
  5. You don't have to.

Writing the script isn't the only outcome of this task. You should execute the script to find bugs in NFD, and file these bugs.

Actions #4

Updated by Hila Ben Abraham about 10 years ago

I want to extend the script to cover set/unset-strategy, destroy face and remove-nexthop.
Let me know if you have any objection.

Actions #5

Updated by Junxiao Shi about 10 years ago

There is currently no way to inspect StrategyChoice table, so strategy-choice/set|unset cannot be reliably tested.

You can add additional functionality, but the Task Description should be updated accordingly to include detailed steps.

Actions #6

Updated by Beichuan Zhang about 10 years ago

Junxiao Shi wrote:

There is currently no way to inspect StrategyChoice table, so strategy-choice/set|unset cannot be reliably tested.

Why is that? Is it because nfd-status doesn't obtain the strategy information from NFD?

Actions #7

Updated by Junxiao Shi about 10 years ago

There isn't a StrategyChoice dataset, so there's no way to directly observe the entries in StrategyChoice table.

However, one may be able to guess the effective strategy by observing the behavior difference between strategies.

Actions #8

Updated by Beichuan Zhang about 10 years ago

Junxiao Shi wrote:

There isn't a StrategyChoice dataset, so there's no way to directly observe the entries in StrategyChoice table.

So this should be added in future versions?

Actions #9

Updated by Hila Ben Abraham about 10 years ago

step 17: don't we need to start pingServer on host B to see a change in host B counters (stage 19)? and if so, don't we have to add ndn:/test-nfdc route also in host B (using the faces created on the remote in 3,5,7,9)

step 19: I can only see the tcp4 face in B. udp4 face is missing. Is that a bug?

Actions #10

Updated by Junxiao Shi about 10 years ago

step 17: don't we need to start pingServer on host B to see a change in host B counters (stage 19)? and if so, don't we have to add ndn:/test-nfdc route also in host B (using the faces created on the remote in 3,5,7,9)

No. Given BroadcastStrategy, Interests should still arrive at B even without a ping-server, but they won't be answered.

step 19: I can only see the tcp4 face in B. udp4 face is missing. Is that a bug?

Bug #1406

Actions #11

Updated by Hila Ben Abraham almost 10 years ago

Bug #1406 is closed, but I still can't see udp4 face in B. Should I reopen this bug?

Actions #12

Updated by Anonymous almost 10 years ago

I improperly closed #1406 (i.e. the submitter is supposed to close it after confirming the merged patch solves the problem), but I can't reproduce it with the latest code.

What do you mean by "can't see udp4 face in B"? I'm running nfd-status on both machines (setup from #1406 with traffic clients) and it shows the face on both ends.

Actions #13

Updated by Hila Ben Abraham almost 10 years ago

In step 19, I can't see the udp4 face created in step 7.

The topology is A(192.168.1.1) -- B(192.168.1.2)

on A:

$ build/bin/nfdc create udp://192.168.1.2 

on B:

$ build/bin/nfd-status 

new udp4 face to 192.168.1.1 is missing.

Actions #14

Updated by Alex Afanasyev almost 10 years ago

This is normal for UDP. To see a face on B, you should send at least one interest towards this face, otherwise B is unaware of the tunnel.

Actions #15

Updated by Hila Ben Abraham almost 10 years ago

Thanks Alex, that makes sense.
However, step #17 should send the interests over this face. I suspect I have another problem hiding here (probably related to my script). I'll investigate that.

Actions #16

Updated by Hila Ben Abraham almost 10 years ago

OK. I think I found a problem in my script that hide another bug:

My script skipped step #16. so the interests in step #17 were sent only on the tcp face which had lower cost (Does "best-route" is the default strategy?) this caused the missing udp4 face in the remote B.

The problem now is that whenever I do set the strategy to use broadcast, ndn-tlv-ping causes nfd to crash with the next error:

nfd: ../daemon/table/strategy-choice.cpp:172: nfd::fw::Strategy& nfd::StrategyChoice::findEffectiveStrategy(const nfd::pit::Entry&) 
const: Assertion `static_cast<bool>(nameTreeEntry)' failed.

I think it's a new problem since I remember it used to worked before.

Should I open a new bug?

Actions #17

Updated by Alex Afanasyev almost 10 years ago

is it with the latest code? I remember John reported similar problem and it was fixed...

Actions #18

Updated by Hila Ben Abraham almost 10 years ago

  • % Done changed from 0 to 70

Update:

  1. All steps were tested again today and ended successfully (with the limitation of IPV6 - see #1426)
  2. Steps 18 & 19 show exactly 100 in/out interests (why the requirement ask for more than 60?).

I'll move on to finalize the script.

Actions #19

Updated by Junxiao Shi almost 10 years ago

  • Feature #1426 is unrelated to IPv6.
  • Requiring at least 60 packets makes the test case more robust, because underlying transport may be unreliable and could have packet loss.
Actions #20

Updated by Hila Ben Abraham almost 10 years ago

sorry, I had a typo. I meant #1428.

Actions #21

Updated by Hila Ben Abraham almost 10 years ago

What about the new nfdc commands (register & unregister)? Do we want to add them to this test?

Actions #22

Updated by Junxiao Shi almost 10 years ago

  • Subject changed from Test nfdc and nfd-status to nfdc and nfd-status test scenario
  • Description updated (diff)
Actions #23

Updated by Alex Afanasyev almost 10 years ago

  • Target version changed from v0.1 to v0.2
Actions #24

Updated by Hila Ben Abraham almost 10 years ago

  • Status changed from In Progress to Code review
  • % Done changed from 70 to 100
Actions #25

Updated by Junxiao Shi almost 10 years ago

uploading patchset 5 verification result

Actions #26

Updated by Junxiao Shi almost 10 years ago

uploading patchset 6 verification result

Actions #27

Updated by Junxiao Shi almost 10 years ago

uploading patchset 7 verification result

Actions #28

Updated by Junxiao Shi almost 10 years ago

uploading patchset 8 verification result

Actions #29

Updated by Hila Ben Abraham almost 10 years ago

  • Status changed from Code review to Closed
Actions

Also available in: Atom PDF