NFD

In the Vagrant environment, this script terminates when running all tests together (test_all). I have not tried when running the single test, but I plan to test this shortly.

I am unable to replicate this issue in the Vagrant environment.

Junxiao, what particular environment are you encountering this issue on? Emulab?

what particular environment are you encountering this issue on?

It’s a node on a private Emulab system.

Cause of this issue is ProcessManager.killProcess is using the wrong PID to kill process.

def killProcess(self, processKey):
    if processKey not in self.results and processKey in self.subprocesses:
        subprocess.call(['sudo', 'kill', str(self.subprocesses[processKey].pid)])

When a process is started, Python's subprocess module assigns self.subprocesses[processKey].pid to be the PID of started process.
Since NFD daemonizes itself, the PID does not match nfd process.
I confirmed this by inserting print to Python code and comparing with pgrep.

To fix this issue, ProcessManager.startNfd and ProcessManager.killNfd should use nfd-start and nfd-stop scripts.

Junxiao Shi wrote:

Since NFD daemonizes itself, the PID does not match nfd process.

What are you talking about? NFD does not daemonize itself.

Junxiao, what I believe you're seeing in the issue description are two threads of nfd (in the curly braces). I believe (and may be wrong) that terminating the parent process would terminate both threads and that the existing code should work to terminate NFD.

Another thought: Perhaps kill is terminating sudo, leaving the nfd process as an orphan.

Eric Newberry wrote:

Another thought: Perhaps kill is terminating sudo, leaving the nfd process as an orphan.

Yes, kill is signaling the sudo process, but sudo should propagate the signal to its child process.

We could also just rewrite this test case as a Bash script, like most other test cases. These other tests cases do similar things, but do not seem to encounter this issue.

We could also just rewrite this test case as a Bash script, like most other test cases.

After all tests moved to bash, there’s no need for a Python wrapper for each test. Use a bash script to invoke each test instead.

These other tests cases do similar things, but do not seem to encounter this issue.

They use nfd-stop or killall nfd mostly, as they should have.

I ran each test individually on the Vagrant environment and found that the following ones failed to terminate:

• test_interest_aggregation
• test_ndnpeekpoke
• test_ndnping
• test_ndntraffic

I pushed a change to rewrite test_ndnpeekpoke, test_ndnping, and test_ndntraffic as Bash-based tests, which resolves this issue. I decided to leave test_interest_aggregation to #4379, since it's broken anyway.

Junxiao Shi wrote:

Change 4833,3 fails to terminate in ./run-vagrant-tests.sh.

Node A has the following process when stuck. Test proceeds after executing nfd-stop in node A.

vagrant@vagrant:~/integration-tests$ pstree -p 2694
run_tests.py(2694)---run_tests.py(5858)---sudo(5861)---nfd(5863)-+-{nfd}(5864)
                                                                 `-{nfd}(5865)

As I said in note 18, I'm not planning to fix test_interest_aggregation, so this is probably the cause.

Junxiao Shi wrote:

I also see error messages during execution:

../permanent-face-test.sh: line 34: [[: 0
0: syntax error in expression (error token is "0")
./permanent-face-test.sh: line 52: [[: 100
0: syntax error in expression (error token is "0")
./permanent-face-test.sh: line 74: [[: 0
10: syntax error in expression (error token is "10")

Fixing the test these are occurring in is not part of this issue, but rather #4379.

Fixing the test these are occurring in is not part of this issue, but rather #4379.

I'm not judging which issue the error messages belong. I'm stating a fact of the appearance of these error messages, just like Jenkins fails the build whenever a test case fails regardless of whether it relates to the current commit.