Bug #3750
closed
xUnit test reports broken on Ubuntu slaves
Added by Davide Pesavento over 7 years ago.
Updated over 7 years ago.
Category:
Integration Tests
Description
It's simply not working. Nothing is published on the build page (e.g. http://jenkins.named-data.net/job/NFD/4165/OS=Ubuntu-14.04-64bit/), and the console output shows xUnit-related errors towards the end.
[xUnit] [INFO] - Starting to record.
[xUnit] [INFO] - Processing BoostTest-1.x (default)
[xUnit] [INFO] - [BoostTest-1.x (default)] - No test report file(s) were found with the pattern 'build/xunit*.xml' relative to '/home/jenkins/workspace/NFD/OS/Ubuntu-14.04-64bit' for the testing framework 'BoostTest-1.x (default)'. Did you enter a pattern relative to the correct directory? Did you generate the result report(s) for 'BoostTest-1.x (default)'?
[xUnit] [WARNING] - No test reports found for the metric 'BoostTest' with the resolved pattern 'build/xunit*.xml'.
[xUnit] [INFO] - Skipping the metric tool processing.
[xUnit] [INFO] - There are errors when processing test results.
[xUnit] [INFO] - Skipping tests recording.
This issue affects all Ubuntu slaves. OSX slaves do not seem to be affected.
- Status changed from New to In Progress
The current Jenkins scripts run the unit tests with the "--log_sink2" option. However, I could find no documentation of this option and this command produced no output to the reference file when I ran it manually. When I removed the '2', changing the option to "--log_sink" (which is documented), the output appears to be produced correctly.
It appears that the "--log_format2" option needs to be changed to "--log_format" as well. I'm currently compiling a list of projects affected by this issue.
That doesn't explain why OS X slaves are unaffected...
Davide Pesavento wrote:
Is $XUNIT set in the jenkins environment? (https://github.com/named-data/NFD/blob/master/.jenkins.d/20-tests.sh#L28)
The logging options are only included on the command line if that's the case. It's set in the build shell like so:
XUNIT=1 ./.jenkins
I looked at the help for the unit tests (unit-tests-core --help) and both --log_sink2 and --log_format2 are not listed. As such, I don't believe our customizations are being compiled into the tests on Ubuntu.
@Alex Could you post the help message from one of the OS X nodes for comparison?
Eric Newberry wrote:
I looked at the help for the unit tests (unit-tests-core --help) and both --log_sink2 and --log_format2 are not listed.
That's expected I think, only Boost.Test built-in options are shown in --help output. However, note the following behavior:
davide@ndnbox:~/NFD$ ./build/unit-tests-core --log_sink2
ERROR: the required argument for option '--log_sink2' is missing
--log_format2 arg Type of second log formatter: HRF or XML
--log_sink2 arg (=-) Second log sink, - for stdout
Test setup error: std::runtime_error: test module initialization failed
davide@ndnbox:~/NFD$ ./build/unit-tests-core -- --log_sink2
Running 63 test cases...
[...]
- Assignee changed from Eric Newberry to Alex Afanasyev
Seems to be a regression introduced in NFD commit 3152abac673382b13c39e7df46d830ec9abfc3e0. Other projects are very likely affected as well. Reassigning to Alex, author of the commit.
- Related to Bug #3417: Boost 1.60 breaks secondary log output in unit tests added
Huh.. For Boost < 1.60, we need to move -- in the command line :(. How did I not notice it in the original issue.
Should we make a fix as a separate commit(s)?
Alex Afanasyev wrote:
Huh.. For Boost < 1.60, we need to move -- in the command line :(. How did I not notice it in the original issue.
meh, do we need yet another conditional in the CI script? is there no way to have a command line compatible with all versions?
Should we make a fix as a separate commit(s)?
I'm confused, separate from what?
Huh.. For Boost < 1.60, we need to move -- in the command line :(. How did I not notice it in the original issue.
meh, do we need yet another conditional in the CI script? is there no way to have a command line compatible with all versions?
I thought -- would be a compatible syntax. But from what I see, the old version, simply ignores everything after -- and the new version will complain for any non-standard parameter before --
Should we make a fix as a separate commit(s)?
I'm confused, separate from what?
Separate from https://gerrit.named-data.net/#/c/3079/
So, the good part of the new change is that we would not need any custom code to have multiple output types. The bad part that we will need 3 customized ways to run the unit tests... One for old boost, one for boost 1.60-1.61, and finally for boost >1.62.
I'll try to make a try for this.
- Status changed from In Progress to Code review
- Target version set to v0.5
- % Done changed from 0 to 70
- Status changed from Code review to Closed
- % Done changed from 70 to 100
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by