Task #1388
closed
nfdc and nfd-status test scenario
Added by Yi Huang about 10 years ago.
Updated almost 10 years ago.
Category:
Integration Tests
Description
Topology: A-B connected on same IPv4 subnet and same IPv6 subnet. Both are NDN nodes.
Script is invoked on host A. Script should control other hosts over ssh.
multi-host.conf has these variables defined: CTRL_B, IP4_A1, IP6_A1, IP4_B1, IP6_B1
nfdc create / add-nexthop / destroy test case
- Start NFD on host A and B.
Unless otherwise noted, perform all steps below on host A.
- Execute nfd-status. Check the existence of udp4/udp6/ether multicast faces.
- Invoke
nfdc create tcp://$IP4_B1. Pause 2 seconds for connection establishment.
- Execute nfd-status. Check the existence of new tcp4 face.
- Invoke
nfdc create tcp://[$IP6_B1]. Pause 2 seconds for connection establishment.
- Execute nfd-status. Check the existence of new tcp6 face.
- Invoke
nfdc create udp://$IP4_B1. Pause 2 seconds for connection establishment.
- Execute nfd-status. Check the existence of new udp4 face.
- Invoke
nfdc create udp://[$IP6_B1]. Pause 2 seconds for connection establishment.
- Execute nfd-status. Check the existence of new udp6 face.
- Invoke nfdc to add a nexthop for
ndn:/test-nfdc with cost 24 toward FaceId created in step 3.
- Invoke nfdc to add a nexthop for
ndn:/test-nfdc with cost 26 toward FaceId created in step 5.
- Invoke nfdc to add a nexthop for
ndn:/test-nfdc with cost 14 toward FaceId created in step 7.
- Invoke nfdc to add a nexthop for
ndn:/test-nfdc with cost 16 toward FaceId created in step 9.
- Execute nfd-status. Check the existence of
ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
- Invoke nfdc to choose BroadcastStrategy for
ndn:/ namespace.
- Invoke ndn-tlv-ping to send 100 Interests under
ndn:/test-nfdc prefix.
Pause 50ms before sending each Interest.
- Execute nfd-status. Verify NOutInterests counters of faces created in step 3,5,7,9 are greater than 60.
- On host B, execute nfd-status. Check the existence of tcp4, tcp6, udp4, udp6 faces with host A's address. Verify NInInterests counters for those faces are greater than 60.
- Invoke nfdc to destroy FaceIds created in step 3,5.
- Execute nfd-status. Check the non-existence of tcp4 and tcp6 faces.
nfdc add-nexthop / remove-nexthop test case
- Start NFD on host A and B.
Unless otherwise noted, perform all steps below on host A.
- Invoke
nfdc add-nexthop -c 44 ndn:/test-nfdc udp4://$IP4_B1. Pause 2 seconds for connection establishment.
- Execute nfd-status. Check the existence of new tcp4 face.
Check the existence of
ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
- Invoke nfdc to remove the nexthop for FaceId seen in step 3.
- Execute nfd-status. Check the non-existence of
ndn:/test-nfdc FIB entry.
nfdc register / unregister test case
- Start NFD on host A and B.
Unless otherwise noted, perform all steps below on host A.
- Invoke
nfdc register -c 55 ndn:/test-nfdc udp4://$IP4_B1. Pause 2 seconds for connection establishment.
- Execute nfd-status. Check the existence of new tcp4 face.
Check the existence of
ndn:/test-nfdc FIB entry, and the correctness of cost on nexthop records.
- Invoke nfdc to unregister the prefix
ndn:/test-nfdc for FaceId seen in step 3.
- Execute nfd-status. Check the non-existence of
ndn:/test-nfdc FIB entry.
Script can assume integrated test suite is cloned to all hosts, necessary programs are installed and accessible in $PATH, and current user can ssh to other hosts and has sudo privilege.
Files
- Subject changed from Test nfdc to Test nfdc and nfd-status
- Description updated (diff)
- Start date deleted (
03/22/2014)
- Estimated time set to 5.00 h
- Status changed from New to In Progress
I have few technical questions:
- What is the right path to store this script? I thought tools/tests might be better than tests/, but I'm open for your suggestions.
- I'm gonna go for bash or python unless you have any other recommendation.
- return value/ printout: Are we going to use it as part of a regression test? if so, is there any return value / printout I should design the script accordingly? or is it sufficient to print something like "FAIL" or "SUCCESS" for now?
- Should the script exit for the first failure or should we run it through the end and provide one output log?
- nfdc exits with a success or failure message, should I check this before trying nfd-status?
- Integrated tests are stored in a separate repository which should be created during Task #1220.
- I believe any language is fine.
- Return zero and print nothing on success. Return non-zero and print something to stdout on failure.
- Either way is fine.
- You don't have to.
Writing the script isn't the only outcome of this task. You should execute the script to find bugs in NFD, and file these bugs.
I want to extend the script to cover set/unset-strategy, destroy face and remove-nexthop.
Let me know if you have any objection.
There is currently no way to inspect StrategyChoice table, so strategy-choice/set|unset cannot be reliably tested.
You can add additional functionality, but the Task Description should be updated accordingly to include detailed steps.
Junxiao Shi wrote:
There is currently no way to inspect StrategyChoice table, so strategy-choice/set|unset cannot be reliably tested.
Why is that? Is it because nfd-status doesn't obtain the strategy information from NFD?
There isn't a StrategyChoice dataset, so there's no way to directly observe the entries in StrategyChoice table.
However, one may be able to guess the effective strategy by observing the behavior difference between strategies.
Junxiao Shi wrote:
There isn't a StrategyChoice dataset, so there's no way to directly observe the entries in StrategyChoice table.
So this should be added in future versions?
step 17: don't we need to start pingServer on host B to see a change in host B counters (stage 19)? and if so, don't we have to add ndn:/test-nfdc route also in host B (using the faces created on the remote in 3,5,7,9)
step 19: I can only see the tcp4 face in B. udp4 face is missing. Is that a bug?
step 17: don't we need to start pingServer on host B to see a change in host B counters (stage 19)? and if so, don't we have to add ndn:/test-nfdc route also in host B (using the faces created on the remote in 3,5,7,9)
No. Given BroadcastStrategy, Interests should still arrive at B even without a ping-server, but they won't be answered.
step 19: I can only see the tcp4 face in B. udp4 face is missing. Is that a bug?
Bug #1406
Bug #1406 is closed, but I still can't see udp4 face in B. Should I reopen this bug?
I improperly closed #1406 (i.e. the submitter is supposed to close it after confirming the merged patch solves the problem), but I can't reproduce it with the latest code.
What do you mean by "can't see udp4 face in B"? I'm running nfd-status on both machines (setup from #1406 with traffic clients) and it shows the face on both ends.
In step 19, I can't see the udp4 face created in step 7.
The topology is A(192.168.1.1) -- B(192.168.1.2)
on A:
$ build/bin/nfdc create udp://192.168.1.2
on B:
$ build/bin/nfd-status
new udp4 face to 192.168.1.1 is missing.
This is normal for UDP. To see a face on B, you should send at least one interest towards this face, otherwise B is unaware of the tunnel.
Thanks Alex, that makes sense.
However, step #17 should send the interests over this face. I suspect I have another problem hiding here (probably related to my script). I'll investigate that.
OK. I think I found a problem in my script that hide another bug:
My script skipped step #16. so the interests in step #17 were sent only on the tcp face which had lower cost (Does "best-route" is the default strategy?) this caused the missing udp4 face in the remote B.
The problem now is that whenever I do set the strategy to use broadcast, ndn-tlv-ping causes nfd to crash with the next error:
nfd: ../daemon/table/strategy-choice.cpp:172: nfd::fw::Strategy& nfd::StrategyChoice::findEffectiveStrategy(const nfd::pit::Entry&)
const: Assertion `static_cast<bool>(nameTreeEntry)' failed.
I think it's a new problem since I remember it used to worked before.
Should I open a new bug?
is it with the latest code? I remember John reported similar problem and it was fixed...
- % Done changed from 0 to 70
Update:
- All steps were tested again today and ended successfully (with the limitation of IPV6 - see #1426)
- Steps 18 & 19 show exactly 100 in/out interests (why the requirement ask for more than 60?).
I'll move on to finalize the script.
- Feature #1426 is unrelated to IPv6.
- Requiring at least 60 packets makes the test case more robust, because underlying transport may be unreliable and could have packet loss.
sorry, I had a typo. I meant #1428.
What about the new nfdc commands (register & unregister)? Do we want to add them to this test?
- Subject changed from Test nfdc and nfd-status to nfdc and nfd-status test scenario
- Description updated (diff)
- Target version changed from v0.1 to v0.2
- Status changed from In Progress to Code review
- % Done changed from 70 to 100
uploading patchset 5 verification result
uploading patchset 6 verification result
uploading patchset 7 verification result
uploading patchset 8 verification result
- Status changed from Code review to Closed
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by 
Updated by Anonymous 
Updated by