Project

General

Profile

Actions

Feature #1895

closed

Clone from alternate code branch in integration tests

Added by Junxiao Shi over 10 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Low
Assignee:
Category:
Integration Tests
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
1.50 h

Description

In IntegrationTests, allow code to be cloned from an alternate branch.

Currently, integration test suite clones ndn-cxx, NFD, and other project code from GitHub master branch.
It's desirable to allow overriding the branch (eg. cloning from a pending Change on Gerrit, or using a local code repository), so that integration tests can be used to evaluate unmerged code.


Related issues 1 (0 open1 closed)

Blocks NFD - Task #4380: Run integration tests for every Jenkins buildAbandoned

Actions
Actions #1

Updated by Junxiao Shi over 10 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Junxiao Shi over 10 years ago

  • Status changed from In Progress to New
  • Assignee deleted (Junxiao Shi)
  • Priority changed from Normal to Low
  • Target version changed from v0.2 to v0.3

I realize that the code branch is controlled by install_helpers/*.py, so I can edit those scripts to checkout an alternate branch.

It is useful to make it configurable by a configuration file, but this is not urgent.

Actions #3

Updated by Junxiao Shi over 10 years ago

  • Tracker changed from Task to Feature
  • Target version deleted (v0.3)

This feature is "good to have". I'm clearing "Target Version" but retaining the Issue in case someone volunteers to work on it.

Actions #4

Updated by Junxiao Shi about 10 years ago

  • Description updated (diff)
  • Target version set to v0.4

From a comment in http://gerrit.named-data.net/1787:

However, there is a real need to run all available integration tests BEFORE the merge.

I suggest creating a repository.conf configuration file with the following format:

PATCHSET_NDNCXX=8888,1
PATCHSET_NFD=master

This indicates that integration tests should use ndn-cxx Change 8888 patchset 1, and NFD master branch.

The following snippet can construct a correct branch name and clone from Gerrit:

GERRIT_CHANGE=$(echo $PATCHSET_NDNCXX | cut -d, -f1)
GERRIT_PATCHSET=$(echo $PATCHSET_NDNCXX | cut -d, -f2)
if [[ $GERRIT_CHANGE == 'master' ]]; then
  GERRIT_BRANCH=master
else
  GERRIT_BRANCH=refs/changes/$(python -c "print('%02d' % (${GERRIT_CHANGE} % 100))")/${GERRIT_CHANGE}/${GERRIT_PATCHSET}
fi

mkdir -p repos/ndn-cxx
cd repos/ndn-cxx

echo Checking out from Gerrit: ndn-cxx $GERRIT_BRANCH
git init
git fetch --depth=1 http://gerrit.named-data.net/ndn-cxx $GERRIT_BRANCH && git checkout FETCH_HEAD
Actions #5

Updated by Junxiao Shi almost 10 years ago

  • Description updated (diff)

I did a major upgrade for ndn-cxx-breaks, and it now has the ability to run integration tests with any patchset of ndn-cxx, NFD, ndn-tools.

It works by modifying the git clone command line Python scripts after cloning integration-tests repository.

Actions #6

Updated by Junxiao Shi almost 10 years ago

At 20150803 conference call, Alex believes this feature is still useful for Vagrant runtime.

Eric says he cannot take this issue because he doesn't know Python enough.

Actions #7

Updated by Davide Pesavento over 8 years ago

  • Target version deleted (v0.4)
Actions #8

Updated by Eric Newberry over 7 years ago

  • Status changed from New to In Progress
  • Assignee set to Eric Newberry
Actions #9

Updated by Eric Newberry over 7 years ago

  • Blocks Task #4380: Run integration tests for every Jenkins build added
Actions #10

Updated by Eric Newberry over 7 years ago

  • Status changed from In Progress to Code review
  • % Done changed from 0 to 100

I also added the ability to provide local versions of packages by specifying "local" as the patchset. This will look for the package in a named subdirectory of the integration test folder (e.g., "NFD" for NFD and "ndn-cxx" for ndn-cxx).

Actions #11

Updated by Junxiao Shi over 7 years ago

Change 4368,9 test log:

/tmp/integ$ grep -v '#' repos.conf

PACKAGE_NDNCXX=gerrit:4381,5
PACKAGE_NFD=local:../NFD
PACKAGE_NDNTOOLS=branch:master
PACKAGE_NDNTRAFFIC=branch:master
PACKAGE_REPONG=branch:master

/tmp/integ$ git -C ../NFD log | head -1
commit 00d39fda3c24e48fc0fee5f0749bcf596f3876dc

Output snippet:

    template: INSTALLING ndn-cxx
    template: ******************
    template: Checking out from Gerrit: master
    template: Initialized empty Git repository in /home/vagrant/nfd_prepare/ndn-cxx/.git/
    template: From https://gerrit.named-data.net/ndn-cxx
    template:  * branch            master     -> FETCH_HEAD
    template: Note: checking out 'FETCH_HEAD'.
    template:
    template: You are in 'detached HEAD' state. You can look around, make experimental
    template: changes and commit them, and you can discard any commits you make in this
    template: state without impacting any branches by performing another checkout.
    template:
    template: If you want to create a new branch to retain commits you create, you may
    template: do so (now or later) by using -b with the checkout command again. Example:
    template:
    template:   git checkout -b new_branch_name
    template: HEAD is now at 7f01247... mgmt: encode/decode CS Information Dataset
[omitted]
    template: 'install' finished successfully (0.476s)
    template: CONFIGURING SECURITY ENVIRONMENT WITH ndnsec
    template: ********************************************
    template: OK: certificate with name [
    template: /tmp/nfd_integration_tests/KEY/%0E%AA%90%B3%B2%88_%8D/self/%FD%00%00%01%60o%E1%99w
    template: ]
    template: has been successfully installed
    template: INSTALLING NFD
    template: **************
    template: Checking out from Gerrit: master
    template: Initialized empty Git repository in /home/vagrant/nfd_prepare/NFD/.git/
    template: From https://gerrit.named-data.net/NFD
    template:  * branch            master     -> FETCH_HEAD
    template: Note: checking out 'FETCH_HEAD'.
    template:
    template: You are in 'detached HEAD' state. You can look around, make experimental
    template: changes and commit them, and you can discard any commits you make in this
    template: state without impacting any branches by performing another checkout.
    template:
    template: If you want to create a new branch to retain commits you create, you may
    template: do so (now or later) by using -b with the checkout command again. Example:
    template:
    template:   git checkout -b new_branch_name
    template:
    template: HEAD is now at 00d39fd... tests: improve LpReliability unit tests
[omitted]
    template: 'install' finished successfully (0.292s)
    template:

I've checked that repos.conf is copied to the template box, but ../NFD is not transferred properly. Neither PACKAGE_NDNCXX nor PACKAGE_NFD directive was effective.

Actions #12

Updated by Eric Newberry over 7 years ago

The issue reported in note 11 should be fixed in the latest patchset.

Actions #13

Updated by Junxiao Shi over 7 years ago

  • Target version set to v0.7

Change 4368,15 test log:

/tmp/integ$ cat repos.conf
PACKAGE_NDNCXX=gerrit:4381,5
PACKAGE_NFD=local:../NFD
PACKAGE_NDNTOOLS=branch:master
PACKAGE_NDNTRAFFIC=branch:master
PACKAGE_REPONG=branch:master

/tmp/integ$ ./install_apps.py install_all
[omitted]
INSTALLING ndn-cxx
******************
Checking out from Gerrit: refs/changes/81/4381/5
Initialized empty Git repository in /users/jns23/nfd_prepare/ndn-cxx/.git/
remote: Counting objects: 770, done
remote: Finding sources: 100% (770/770)
remote: Total 770 (delta 53), reused 236 (delta 53)
Receiving objects: 100% (770/770), 1.14 MiB | 0 bytes/s, done.
Resolving deltas: 100% (53/53), done.
From https://gerrit.named-data.net/ndn-cxx
 * branch            refs/changes/81/4381/5 -> FETCH_HEAD
[omitted]
INSTALLING NFD
**************
Using local copy of NFD from ../NFD
install_helpers/common.sh: line 39: cd: /users/jns23/nfd_prepare/NFD: No such file or directory
./install_helpers/install_NFD.sh: line 11: ./waf: No such file or directory
./install_helpers/install_NFD.sh: line 12: ./waf: No such file or directory
sudo: ./waf: command not found
cp: cannot stat '/usr/local/etc/ndn/nfd.conf.sample': No such file or directory
[omitted]

I am NOT using Vagrant. ./install_apps.py is one of several methods to run integration tests, as described in README.md.

In case you are wondering, verification is passing on Vagrant.

Actions #14

Updated by Eric Newberry over 7 years ago

Junxiao Shi wrote:

I am NOT using Vagrant. ./install_apps.py is one of several methods to run integration tests, as described in README.md.

In case you are wondering, verification is passing on Vagrant.

We still need some way to copy a local repository to a location accessible from the testing machines (VM or otherwise), which will vary by platform. I assume you're using Emulab to run the tests above? Perhaps you could write a script that does what run-vagrant-tests.sh does with any local repositories (creates a folder nfd_prepare in the integration_tests folder and then copies the provided local repository folders to that location).

Actions #15

Updated by Junxiao Shi over 7 years ago

We still need some way to copy a local repository to a location accessible from the testing machines (VM or otherwise), which will vary by platform.

In my test, the local repository is already available on the testing machine (node A).

I assume you're using Emulab to run the tests above?

No. It is a single machine, intending to run a single host test case such as test_ndnping.

The problem is: repos.conf claims to support “local:../NFD” syntax, but ./install_apps.py is looking for the repository in nfd_prepare. It is wrong to put the copying logic in Vagrant script, because Vagrant is not the only way to run tests. ./install_apps.py needs to looks at the directory specified in repos.conf.
When Vagrant is used, the Vagrant script can upload the local repository, and upload a modified version of repos.conf to point to its location on the template machine, or simply let ./install_apps.py check both locations.

Actions #16

Updated by Junxiao Shi over 7 years ago

Change 4368,17 test log:

$ cat repos.conf
PACKAGE_NDNCXX=gerrit:4381,5
PACKAGE_NFD=local:../NFD

$ git -C ../NFD log | head -7
commit 4738f15594d7a1c62f692f7fb2a372aabed9308e
Author: Eric Newberry <enewberry@cs.arizona.edu>
Date:   Mon Dec 18 13:10:32 2017 -0700

    tests: make CongestionMarkStrategy parameters human-readable

    Change-Id: I45448640b9125015a0ae2711f14fc3c1472603c3

Output snippet of ./run-vagrant-tests.sh:

    template: INSTALLING ndn-cxx
    template: ******************
    template: Checking out from Gerrit: refs/changes/81/4381/5
    template: Initialized empty Git repository in /home/vagrant/integration-tests/prepare/ndn-cxx/.git/
    template: From https://gerrit.named-data.net/ndn-cxx
    template:  * branch            refs/changes/81/4381/5 -> FETCH_HEAD

    template: INSTALLING NFD
    template: **************
    template: Using local copy of NFD
    template: ERROR: No local copy of package NFD could be found in prepare
    template:
Actions #17

Updated by Eric Newberry over 7 years ago

  • Status changed from Code review to Closed
Actions

Also available in: Atom PDF