Project

General

Profile

Actions

Task #1948

open

Run integrated tests for every Jenkins build

Added by Alex Afanasyev about 10 years ago. Updated over 6 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Build
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
4.50 h

Description

Run ndn-cxx integrated tests as part of Jenkins build procedure.
NFD should be built and started during the execution of those tests.


Related issues 3 (0 open3 closed)

Related to ndn-cxx - Task #1175: Separate tests into unit-tests and integration testsClosed

Actions
Related to NFD - Task #4380: Run integration tests for every Jenkins buildAbandoned

Actions
Blocked by ndn-cxx - Bug #3684: Face integ: "child has exited"ClosedJunxiao Shi07/25/2016

Actions
Actions #1

Updated by Junxiao Shi about 10 years ago

  • Related to Task #1175: Separate tests into unit-tests and integration tests added
Actions #2

Updated by Junxiao Shi about 10 years ago

  • Start date deleted (08/28/2014)

This Task by itself isn't difficult: add a step to build NFD and run integration tests.

The primary concern is: some ndn-cxx Change is breaking, and it requires an accompanying NFD Change.

It's unfeasible to require every ndn-cxx Change to be backwards compatible.

One solution would be: if a commit is expected to be breaking, add [breaks NFD] to COMMIT_MSG, so that integration-tests are skipped for this particular commit.

Actions #3

Updated by Junxiao Shi almost 10 years ago

20141128 conference call approves the idea in note-2.

The exact token in COMMIT_MSG should be:

Breaks: nfd:commit:b499a601b56b27f69929df64fa6d0f0151b37284

This token should be placed on a separate line, and the commit hash must be one of the last twelve NFD commits.

Actions #4

Updated by Alex Afanasyev almost 9 years ago

  • Target version changed from v0.3 to v0.5
Actions #5

Updated by Junxiao Shi over 8 years ago

  • Blocked by Bug #3684: Face integ: "child has exited" added
Actions #6

Updated by Junxiao Shi almost 8 years ago

  • Assignee set to Eric Newberry
  • Target version changed from v0.5 to v0.6
  • Estimated time set to 4.50 h
Actions #7

Updated by Eric Newberry almost 8 years ago

So is this task to enable the integrated tests built with the code or the NFD integration test suite?

Actions #8

Updated by Junxiao Shi almost 8 years ago

  • Tracker changed from Task to Feature
  • Description updated (diff)

Currently, only tests/integrated/face.cpp needs to be executed.

Actions #9

Updated by Eric Newberry almost 8 years ago

  • Status changed from New to In Progress
Actions #10

Updated by Eric Newberry almost 8 years ago

  • Status changed from In Progress to Code review
  • % Done changed from 0 to 100
Actions #11

Updated by Junxiao Shi almost 8 years ago

disagree. this should not be part of CI script. The integration script should exist as something separate, potentially running as a separate job

I would argue that ndn-cxx integrated tests should be executed in the same job as unit testing, rather than a separate job.
Suppose we use a separate job, it is still necessary to execute integrated tests on every platform. Unlike Doxygen checker, the behavior of integrated tests are not guaranteed to be the same on different platforms.
By using a separate job, duplicate computation resources are spent on compiling ndn-cxx library one more time.

Actions #12

Updated by Alex Afanasyev almost 8 years ago

I'm not sure behavior needs to be checked on every platform. Our (real) integrated tests are running just on Linux platform and do their job pretty well.

The tests in question (integrated face) technically should be part of unit tests suite. However, to make it work, we need to implement a DummyForwarder (MockFowarder, FakeForwarder). Though, not sure how complex/simple this could be.

Actions #13

Updated by Junxiao Shi over 7 years ago

  • Status changed from Code review to New

https://gerrit.named-data.net/3518 Change is abandoned, returning to New state.

Actions #14

Updated by Davide Pesavento about 7 years ago

  • Target version deleted (v0.6)
  • % Done changed from 100 to 0
Actions #15

Updated by Davide Pesavento about 7 years ago

  • Related to Task #4380: Run integration tests for every Jenkins build added
Actions #16

Updated by Davide Pesavento about 7 years ago

  • Tracker changed from Feature to Task
  • Subject changed from Enable integrated tests in Jenkins to Run integrated tests for every Jenkins build
  • Category changed from Tests to Build
Actions #17

Updated by Eric Newberry over 6 years ago

  • Assignee deleted (Eric Newberry)
Actions

Also available in: Atom PDF