Task #1948
open
Run integrated tests for every Jenkins build
0%
Description
Run ndn-cxx integrated tests as part of Jenkins build procedure.
NFD should be built and started during the execution of those tests.
Updated by Junxiao Shi over 9 years ago
- Related to Task #1175: Separate tests into unit-tests and integration tests added
Updated by Junxiao Shi over 9 years ago
- Start date deleted (
08/28/2014)
This Task by itself isn't difficult: add a step to build NFD and run integration tests.
The primary concern is: some ndn-cxx Change is breaking, and it requires an accompanying NFD Change.
It's unfeasible to require every ndn-cxx Change to be backwards compatible.
One solution would be: if a commit is expected to be breaking, add [breaks NFD] to COMMIT_MSG, so that integration-tests are skipped for this particular commit.
Updated by Junxiao Shi over 9 years ago
20141128 conference call approves the idea in note-2.
The exact token in COMMIT_MSG should be:
Breaks: nfd:commit:b499a601b56b27f69929df64fa6d0f0151b37284
This token should be placed on a separate line, and the commit hash must be one of the last twelve NFD commits.
Updated by Alex Afanasyev over 8 years ago
- Target version changed from v0.3 to v0.5
Updated by Junxiao Shi over 7 years ago
- Blocked by Bug #3684: Face integ: "child has exited" added
Updated by Junxiao Shi over 7 years ago
- Assignee set to Eric Newberry
- Target version changed from v0.5 to v0.6
- Estimated time set to 4.50 h
Updated by Eric Newberry over 7 years ago
So is this task to enable the integrated tests built with the code or the NFD integration test suite?
Updated by Junxiao Shi over 7 years ago
- Tracker changed from Task to Feature
- Description updated (diff)
Currently, only tests/integrated/face.cpp needs to be executed.
Updated by Eric Newberry over 7 years ago
- Status changed from New to In Progress
Updated by Eric Newberry over 7 years ago
- Status changed from In Progress to Code review
- % Done changed from 0 to 100
Updated by Junxiao Shi over 7 years ago
disagree. this should not be part of CI script. The integration script should exist as something separate, potentially running as a separate job
I would argue that ndn-cxx integrated tests should be executed in the same job as unit testing, rather than a separate job.
Suppose we use a separate job, it is still necessary to execute integrated tests on every platform. Unlike Doxygen checker, the behavior of integrated tests are not guaranteed to be the same on different platforms.
By using a separate job, duplicate computation resources are spent on compiling ndn-cxx library one more time.
Updated by Alex Afanasyev over 7 years ago
I'm not sure behavior needs to be checked on every platform. Our (real) integrated tests are running just on Linux platform and do their job pretty well.
The tests in question (integrated face) technically should be part of unit tests suite. However, to make it work, we need to implement a DummyForwarder (MockFowarder, FakeForwarder). Though, not sure how complex/simple this could be.
Updated by Junxiao Shi almost 7 years ago
- Status changed from Code review to New
https://gerrit.named-data.net/3518 Change is abandoned, returning to New state.
Updated by Davide Pesavento over 6 years ago
- Target version deleted (
v0.6) - % Done changed from 100 to 0
Updated by Davide Pesavento over 6 years ago
- Related to Task #4380: Run integration tests for every Jenkins build added
Updated by Davide Pesavento over 6 years ago
- Tracker changed from Feature to Task
- Subject changed from Enable integrated tests in Jenkins to Run integrated tests for every Jenkins build
- Category changed from Tests to Build