Project

General

Profile

Actions

Feature #1895

closed

Clone from alternate code branch in integration tests

Added by Junxiao Shi over 9 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Low
Assignee:
Category:
Integration Tests
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
1.50 h

Description

In IntegrationTests, allow code to be cloned from an alternate branch.

Currently, integration test suite clones ndn-cxx, NFD, and other project code from GitHub master branch.
It's desirable to allow overriding the branch (eg. cloning from a pending Change on Gerrit, or using a local code repository), so that integration tests can be used to evaluate unmerged code.


Related issues 1 (0 open1 closed)

Blocks NFD - Task #4380: Run integration tests for every Jenkins buildAbandoned

Actions
Actions #1

Updated by Junxiao Shi over 9 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Junxiao Shi over 9 years ago

  • Status changed from In Progress to New
  • Assignee deleted (Junxiao Shi)
  • Priority changed from Normal to Low
  • Target version changed from v0.2 to v0.3

I realize that the code branch is controlled by install_helpers/*.py, so I can edit those scripts to checkout an alternate branch.

It is useful to make it configurable by a configuration file, but this is not urgent.

Actions #3

Updated by Junxiao Shi over 9 years ago

  • Tracker changed from Task to Feature
  • Target version deleted (v0.3)

This feature is "good to have". I'm clearing "Target Version" but retaining the Issue in case someone volunteers to work on it.

Actions #4

Updated by Junxiao Shi about 9 years ago

  • Description updated (diff)
  • Target version set to v0.4

From a comment in http://gerrit.named-data.net/1787:

However, there is a real need to run all available integration tests BEFORE the merge.

I suggest creating a repository.conf configuration file with the following format:

PATCHSET_NDNCXX=8888,1
PATCHSET_NFD=master

This indicates that integration tests should use ndn-cxx Change 8888 patchset 1, and NFD master branch.

The following snippet can construct a correct branch name and clone from Gerrit:

GERRIT_CHANGE=$(echo $PATCHSET_NDNCXX | cut -d, -f1)
GERRIT_PATCHSET=$(echo $PATCHSET_NDNCXX | cut -d, -f2)
if [[ $GERRIT_CHANGE == 'master' ]]; then
  GERRIT_BRANCH=master
else
  GERRIT_BRANCH=refs/changes/$(python -c "print('%02d' % (${GERRIT_CHANGE} % 100))")/${GERRIT_CHANGE}/${GERRIT_PATCHSET}
fi

mkdir -p repos/ndn-cxx
cd repos/ndn-cxx

echo Checking out from Gerrit: ndn-cxx $GERRIT_BRANCH
git init
git fetch --depth=1 http://gerrit.named-data.net/ndn-cxx $GERRIT_BRANCH && git checkout FETCH_HEAD
Actions #5

Updated by Junxiao Shi almost 9 years ago

  • Description updated (diff)

I did a major upgrade for ndn-cxx-breaks, and it now has the ability to run integration tests with any patchset of ndn-cxx, NFD, ndn-tools.

It works by modifying the git clone command line Python scripts after cloning integration-tests repository.

Actions #6

Updated by Junxiao Shi over 8 years ago

At 20150803 conference call, Alex believes this feature is still useful for Vagrant runtime.

Eric says he cannot take this issue because he doesn't know Python enough.

Actions #7

Updated by Davide Pesavento over 7 years ago

  • Target version deleted (v0.4)
Actions #8

Updated by Eric Newberry over 6 years ago

  • Status changed from New to In Progress
  • Assignee set to Eric Newberry
Actions #9

Updated by Eric Newberry over 6 years ago

  • Blocks Task #4380: Run integration tests for every Jenkins build added
Actions #10

Updated by Eric Newberry over 6 years ago

  • Status changed from In Progress to Code review
  • % Done changed from 0 to 100

I also added the ability to provide local versions of packages by specifying "local" as the patchset. This will look for the package in a named subdirectory of the integration test folder (e.g., "NFD" for NFD and "ndn-cxx" for ndn-cxx).

Actions #11

Updated by Junxiao Shi over 6 years ago

Change 4368,9 test log:

/tmp/integ$ grep -v '#' repos.conf

PACKAGE_NDNCXX=gerrit:4381,5
PACKAGE_NFD=local:../NFD
PACKAGE_NDNTOOLS=branch:master
PACKAGE_NDNTRAFFIC=branch:master
PACKAGE_REPONG=branch:master

/tmp/integ$ git -C ../NFD log | head -1
commit 00d39fda3c24e48fc0fee5f0749bcf596f3876dc

Output snippet:

    template: INSTALLING ndn-cxx
    template: ******************
    template: Checking out from Gerrit: master
    template: Initialized empty Git repository in /home/vagrant/nfd_prepare/ndn-cxx/.git/
    template: From https://gerrit.named-data.net/ndn-cxx
    template:  * branch            master     -> FETCH_HEAD
    template: Note: checking out 'FETCH_HEAD'.
    template:
    template: You are in 'detached HEAD' state. You can look around, make experimental
    template: changes and commit them, and you can discard any commits you make in this
    template: state without impacting any branches by performing another checkout.
    template:
    template: If you want to create a new branch to retain commits you create, you may
    template: do so (now or later) by using -b with the checkout command again. Example:
    template:
    template:   git checkout -b new_branch_name
    template: HEAD is now at 7f01247... mgmt: encode/decode CS Information Dataset
[omitted]
    template: 'install' finished successfully (0.476s)
    template: CONFIGURING SECURITY ENVIRONMENT WITH ndnsec
    template: ********************************************
    template: OK: certificate with name [
    template: /tmp/nfd_integration_tests/KEY/%0E%AA%90%B3%B2%88_%8D/self/%FD%00%00%01%60o%E1%99w
    template: ]
    template: has been successfully installed
    template: INSTALLING NFD
    template: **************
    template: Checking out from Gerrit: master
    template: Initialized empty Git repository in /home/vagrant/nfd_prepare/NFD/.git/
    template: From https://gerrit.named-data.net/NFD
    template:  * branch            master     -> FETCH_HEAD
    template: Note: checking out 'FETCH_HEAD'.
    template:
    template: You are in 'detached HEAD' state. You can look around, make experimental
    template: changes and commit them, and you can discard any commits you make in this
    template: state without impacting any branches by performing another checkout.
    template:
    template: If you want to create a new branch to retain commits you create, you may
    template: do so (now or later) by using -b with the checkout command again. Example:
    template:
    template:   git checkout -b new_branch_name
    template:
    template: HEAD is now at 00d39fd... tests: improve LpReliability unit tests
[omitted]
    template: 'install' finished successfully (0.292s)
    template:

I've checked that repos.conf is copied to the template box, but ../NFD is not transferred properly. Neither PACKAGE_NDNCXX nor PACKAGE_NFD directive was effective.

Actions #12

Updated by Eric Newberry over 6 years ago

The issue reported in note 11 should be fixed in the latest patchset.

Actions #13

Updated by Junxiao Shi over 6 years ago

  • Target version set to v0.7

Change 4368,15 test log:

/tmp/integ$ cat repos.conf
PACKAGE_NDNCXX=gerrit:4381,5
PACKAGE_NFD=local:../NFD
PACKAGE_NDNTOOLS=branch:master
PACKAGE_NDNTRAFFIC=branch:master
PACKAGE_REPONG=branch:master

/tmp/integ$ ./install_apps.py install_all
[omitted]
INSTALLING ndn-cxx
******************
Checking out from Gerrit: refs/changes/81/4381/5
Initialized empty Git repository in /users/jns23/nfd_prepare/ndn-cxx/.git/
remote: Counting objects: 770, done
remote: Finding sources: 100% (770/770)
remote: Total 770 (delta 53), reused 236 (delta 53)
Receiving objects: 100% (770/770), 1.14 MiB | 0 bytes/s, done.
Resolving deltas: 100% (53/53), done.
From https://gerrit.named-data.net/ndn-cxx
 * branch            refs/changes/81/4381/5 -> FETCH_HEAD
[omitted]
INSTALLING NFD
**************
Using local copy of NFD from ../NFD
install_helpers/common.sh: line 39: cd: /users/jns23/nfd_prepare/NFD: No such file or directory
./install_helpers/install_NFD.sh: line 11: ./waf: No such file or directory
./install_helpers/install_NFD.sh: line 12: ./waf: No such file or directory
sudo: ./waf: command not found
cp: cannot stat '/usr/local/etc/ndn/nfd.conf.sample': No such file or directory
[omitted]

I am NOT using Vagrant. ./install_apps.py is one of several methods to run integration tests, as described in README.md.

In case you are wondering, verification is passing on Vagrant.

Actions #14

Updated by Eric Newberry over 6 years ago

Junxiao Shi wrote:

I am NOT using Vagrant. ./install_apps.py is one of several methods to run integration tests, as described in README.md.

In case you are wondering, verification is passing on Vagrant.

We still need some way to copy a local repository to a location accessible from the testing machines (VM or otherwise), which will vary by platform. I assume you're using Emulab to run the tests above? Perhaps you could write a script that does what run-vagrant-tests.sh does with any local repositories (creates a folder nfd_prepare in the integration_tests folder and then copies the provided local repository folders to that location).

Actions #15

Updated by Junxiao Shi over 6 years ago

We still need some way to copy a local repository to a location accessible from the testing machines (VM or otherwise), which will vary by platform.

In my test, the local repository is already available on the testing machine (node A).

I assume you're using Emulab to run the tests above?

No. It is a single machine, intending to run a single host test case such as test_ndnping.

The problem is: repos.conf claims to support “local:../NFD” syntax, but ./install_apps.py is looking for the repository in nfd_prepare. It is wrong to put the copying logic in Vagrant script, because Vagrant is not the only way to run tests. ./install_apps.py needs to looks at the directory specified in repos.conf.
When Vagrant is used, the Vagrant script can upload the local repository, and upload a modified version of repos.conf to point to its location on the template machine, or simply let ./install_apps.py check both locations.

Actions #16

Updated by Junxiao Shi over 6 years ago

Change 4368,17 test log:

$ cat repos.conf
PACKAGE_NDNCXX=gerrit:4381,5
PACKAGE_NFD=local:../NFD

$ git -C ../NFD log | head -7
commit 4738f15594d7a1c62f692f7fb2a372aabed9308e
Author: Eric Newberry <enewberry@cs.arizona.edu>
Date:   Mon Dec 18 13:10:32 2017 -0700

    tests: make CongestionMarkStrategy parameters human-readable

    Change-Id: I45448640b9125015a0ae2711f14fc3c1472603c3

Output snippet of ./run-vagrant-tests.sh:

    template: INSTALLING ndn-cxx
    template: ******************
    template: Checking out from Gerrit: refs/changes/81/4381/5
    template: Initialized empty Git repository in /home/vagrant/integration-tests/prepare/ndn-cxx/.git/
    template: From https://gerrit.named-data.net/ndn-cxx
    template:  * branch            refs/changes/81/4381/5 -> FETCH_HEAD

    template: INSTALLING NFD
    template: **************
    template: Using local copy of NFD
    template: ERROR: No local copy of package NFD could be found in prepare
    template:
Actions #17

Updated by Eric Newberry over 6 years ago

  • Status changed from Code review to Closed
Actions

Also available in: Atom PDF