Project

General

Profile

Actions
Copy link

Task #1388

closed

nfdc and nfd-status test scenario

Added by Yi Huang almost 11 years ago. Updated over 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Hila Ben Abraham
Category:
Integration Tests
Target version:
v0.2
Start date:
Due date:
% Done:

100%

Estimated time:
5.00 h

Description

Topology: A-B connected on same IPv4 subnet and same IPv6 subnet. Both are NDN nodes.

Script is invoked on host A. Script should control other hosts over ssh.

multi-host.conf has these variables defined: CTRL_B, IP4_A1, IP6_A1, IP4_B1, IP6_B1

nfdc create / add-nexthop / destroy test case

  1. Start NFD on host A and B.
    Unless otherwise noted, perform all steps below on host A.
  2. Execute nfd-status. Check the existence of udp4/udp6/ether multicast faces.
  3. Invoke nfdc create tcp://$IP4_B1. Pause 2 seconds for connection establishment.
  4. Execute nfd-status. Check the existence of new tcp4 face.
  5. Invoke nfdc create tcp://[$IP6_B1]. Pause 2 seconds for connection establishment.
  6. Execute nfd-status. Check the existence of new tcp6 face.
  7. Invoke nfdc create udp://$IP4_B1. Pause 2 seconds for connection establishment.
  8. Execute nfd-status. Check the existence of new udp4 face.
  9. Invoke nfdc create udp://[$IP6_B1]. Pause 2 seconds for connection establishment.
  10. Execute nfd-status. Check the existence of new udp6 face.
  11. Invoke nfdc to add a nexthop for ndn:/test-nfdc with cost 24 toward FaceId created in step 3.
  12. Invoke nfdc to add a nexthop for ndn:/test-nfdc with cost 26 toward FaceId created in step 5.
  13. Invoke nfdc to add a nexthop for ndn:/test-nfdc with cost 14 toward FaceId created in step 7.
  14. Invoke nfdc to add a nexthop for ndn:/test-nfdc with cost 16 toward FaceId created in step 9.
  15. Execute nfd-status. Check the existence of ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
  16. Invoke nfdc to choose BroadcastStrategy for ndn:/ namespace.
  17. Invoke ndn-tlv-ping to send 100 Interests under ndn:/test-nfdc prefix.
    Pause 50ms before sending each Interest.
  18. Execute nfd-status. Verify NOutInterests counters of faces created in step 3,5,7,9 are greater than 60.
  19. On host B, execute nfd-status. Check the existence of tcp4, tcp6, udp4, udp6 faces with host A's address. Verify NInInterests counters for those faces are greater than 60.
  20. Invoke nfdc to destroy FaceIds created in step 3,5.
  21. Execute nfd-status. Check the non-existence of tcp4 and tcp6 faces.

nfdc add-nexthop / remove-nexthop test case

  1. Start NFD on host A and B.
    Unless otherwise noted, perform all steps below on host A.
  2. Invoke nfdc add-nexthop -c 44 ndn:/test-nfdc udp4://$IP4_B1. Pause 2 seconds for connection establishment.
  3. Execute nfd-status. Check the existence of new tcp4 face. Check the existence of ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
  4. Invoke nfdc to remove the nexthop for FaceId seen in step 3.
  5. Execute nfd-status. Check the non-existence of ndn:/test-nfdc FIB entry.

nfdc register / unregister test case

  1. Start NFD on host A and B.
    Unless otherwise noted, perform all steps below on host A.
  2. Invoke nfdc register -c 55 ndn:/test-nfdc udp4://$IP4_B1. Pause 2 seconds for connection establishment.
  3. Execute nfd-status. Check the existence of new tcp4 face. Check the existence of ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
  4. Invoke nfdc to unregister the prefix ndn:/test-nfdc for FaceId seen in step 3.
  5. Execute nfd-status. Check the non-existence of ndn:/test-nfdc FIB entry.

Script can assume integrated test suite is cloned to all hosts, necessary programs are installed and accessible in $PATH, and current user can ssh to other hosts and has sudo privilege.

Files

Download all files
result_c858-5.tgz (55.1 KB) result_c858-5.tgz Junxiao Shi, 05/29/2014 03:50 PM
result_c858-6.zip (55.2 KB) result_c858-6.zip Junxiao Shi, 05/29/2014 11:22 PM
result_c858-7.tgz (55.3 KB) result_c858-7.tgz Junxiao Shi, 06/10/2014 02:54 PM
result_c858-8.zip (60.1 KB) result_c858-8.zip Junxiao Shi, 06/12/2014 12:07 AM
Actions
Copy link
 #1

Updated by Junxiao Shi almost 11 years ago

  • Subject changed from Test nfdc to Test nfdc and nfd-status
  • Description updated (diff)
  • Start date deleted (03/22/2014)
  • Estimated time set to 5.00 h
Actions
Copy link
 #2

Updated by Hila Ben Abraham almost 11 years ago

  • Status changed from New to In Progress

I have few technical questions:

  1. What is the right path to store this script? I thought tools/tests might be better than tests/, but I'm open for your suggestions.
  2. I'm gonna go for bash or python unless you have any other recommendation.
  3. return value/ printout: Are we going to use it as part of a regression test? if so, is there any return value / printout I should design the script accordingly? or is it sufficient to print something like "FAIL" or "SUCCESS" for now?
  4. Should the script exit for the first failure or should we run it through the end and provide one output log?
  5. nfdc exits with a success or failure message, should I check this before trying nfd-status?
Actions
Copy link
 #3

Updated by Junxiao Shi almost 11 years ago

  1. Integrated tests are stored in a separate repository which should be created during Task #1220.
  2. I believe any language is fine.
  3. Return zero and print nothing on success. Return non-zero and print something to stdout on failure.
  4. Either way is fine.
  5. You don't have to.

Writing the script isn't the only outcome of this task. You should execute the script to find bugs in NFD, and file these bugs.

Actions
Copy link
 #4

Updated by Hila Ben Abraham over 10 years ago

I want to extend the script to cover set/unset-strategy, destroy face and remove-nexthop.
Let me know if you have any objection.

Actions
Copy link
 #5

Updated by Junxiao Shi over 10 years ago

There is currently no way to inspect StrategyChoice table, so strategy-choice/set|unset cannot be reliably tested.

You can add additional functionality, but the Task Description should be updated accordingly to include detailed steps.

Actions
Copy link
 #6

Updated by Beichuan Zhang over 10 years ago

Junxiao Shi wrote:

There is currently no way to inspect StrategyChoice table, so strategy-choice/set|unset cannot be reliably tested.

Why is that? Is it because nfd-status doesn't obtain the strategy information from NFD?

Actions
Copy link
 #7

Updated by Junxiao Shi over 10 years ago

There isn't a StrategyChoice dataset, so there's no way to directly observe the entries in StrategyChoice table.

However, one may be able to guess the effective strategy by observing the behavior difference between strategies.

Actions
Copy link
 #8

Updated by Beichuan Zhang over 10 years ago

Junxiao Shi wrote:

There isn't a StrategyChoice dataset, so there's no way to directly observe the entries in StrategyChoice table.

So this should be added in future versions?

Actions
Copy link
 #9

Updated by Hila Ben Abraham over 10 years ago

step 17: don't we need to start pingServer on host B to see a change in host B counters (stage 19)? and if so, don't we have to add ndn:/test-nfdc route also in host B (using the faces created on the remote in 3,5,7,9)

step 19: I can only see the tcp4 face in B. udp4 face is missing. Is that a bug?

Actions
Copy link
 #10

Updated by Junxiao Shi over 10 years ago

step 17: don't we need to start pingServer on host B to see a change in host B counters (stage 19)? and if so, don't we have to add ndn:/test-nfdc route also in host B (using the faces created on the remote in 3,5,7,9)

No. Given BroadcastStrategy, Interests should still arrive at B even without a ping-server, but they won't be answered.

step 19: I can only see the tcp4 face in B. udp4 face is missing. Is that a bug?

Bug #1406

Actions
Copy link
 #11

Updated by Hila Ben Abraham over 10 years ago

Bug #1406 is closed, but I still can't see udp4 face in B. Should I reopen this bug?

Actions
Copy link
 #12

Updated by Anonymous over 10 years ago

I improperly closed #1406 (i.e. the submitter is supposed to close it after confirming the merged patch solves the problem), but I can't reproduce it with the latest code.

What do you mean by "can't see udp4 face in B"? I'm running nfd-status on both machines (setup from #1406 with traffic clients) and it shows the face on both ends.

Actions
Copy link
 #13

Updated by Hila Ben Abraham over 10 years ago

In step 19, I can't see the udp4 face created in step 7.

The topology is A(192.168.1.1) -- B(192.168.1.2)

on A:

$ build/bin/nfdc create udp://192.168.1.2

on B:

$ build/bin/nfd-status

new udp4 face to 192.168.1.1 is missing.

Actions
Copy link
 #14

Updated by Alex Afanasyev over 10 years ago

This is normal for UDP. To see a face on B, you should send at least one interest towards this face, otherwise B is unaware of the tunnel.

Actions
Copy link
 #15

Updated by Hila Ben Abraham over 10 years ago

Thanks Alex, that makes sense.
However, step #17 should send the interests over this face. I suspect I have another problem hiding here (probably related to my script). I'll investigate that.

Actions
Copy link
 #16

Updated by Hila Ben Abraham over 10 years ago

OK. I think I found a problem in my script that hide another bug:

My script skipped step #16. so the interests in step #17 were sent only on the tcp face which had lower cost (Does "best-route" is the default strategy?) this caused the missing udp4 face in the remote B.

The problem now is that whenever I do set the strategy to use broadcast, ndn-tlv-ping causes nfd to crash with the next error:

nfd: ../daemon/table/strategy-choice.cpp:172: nfd::fw::Strategy& nfd::StrategyChoice::findEffectiveStrategy(const nfd::pit::Entry&) 
const: Assertion `static_cast<bool>(nameTreeEntry)' failed.

I think it's a new problem since I remember it used to worked before.

Should I open a new bug?

Actions
Copy link
 #17

Updated by Alex Afanasyev over 10 years ago

is it with the latest code? I remember John reported similar problem and it was fixed...

Actions
Copy link
 #18

Updated by Hila Ben Abraham over 10 years ago

  • % Done changed from 0 to 70

Update:

  1. All steps were tested again today and ended successfully (with the limitation of IPV6 - see #1426)
  2. Steps 18 & 19 show exactly 100 in/out interests (why the requirement ask for more than 60?).

I'll move on to finalize the script.

Actions
Copy link
 #19

Updated by Junxiao Shi over 10 years ago

  • Feature #1426 is unrelated to IPv6.
  • Requiring at least 60 packets makes the test case more robust, because underlying transport may be unreliable and could have packet loss.
Actions
Copy link
 #20

Updated by Hila Ben Abraham over 10 years ago

sorry, I had a typo. I meant #1428.

Actions
Copy link
 #21

Updated by Hila Ben Abraham over 10 years ago

What about the new nfdc commands (register & unregister)? Do we want to add them to this test?

Actions
Copy link
 #22

Updated by Junxiao Shi over 10 years ago

  • Subject changed from Test nfdc and nfd-status to nfdc and nfd-status test scenario
  • Description updated (diff)
Actions
Copy link
 #23

Updated by Alex Afanasyev over 10 years ago

  • Target version changed from v0.1 to v0.2
Actions
Copy link
 #24

Updated by Hila Ben Abraham over 10 years ago

  • Status changed from In Progress to Code review
  • % Done changed from 70 to 100
Actions
Copy link
 #25

Updated by Junxiao Shi over 10 years ago

uploading patchset 5 verification result

Actions
Copy link
 #26

Updated by Junxiao Shi over 10 years ago

uploading patchset 6 verification result

Actions
Copy link
 #27

Updated by Junxiao Shi over 10 years ago

uploading patchset 7 verification result

Actions
Copy link
 #28

Updated by Junxiao Shi over 10 years ago

uploading patchset 8 verification result

Actions
Copy link
 #29

Updated by Hila Ben Abraham over 10 years ago

  • Status changed from Code review to Closed
Actions
Copy link

Also available in: Atom PDF