Project

General

Profile

Actions

Bug #3750

closed

xUnit test reports broken on Ubuntu slaves

Added by Davide Pesavento over 8 years ago. Updated about 8 years ago.

Status:
Closed
Priority:
Normal
Category:
Integration Tests
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:

Description

It's simply not working. Nothing is published on the build page (e.g. http://jenkins.named-data.net/job/NFD/4165/OS=Ubuntu-14.04-64bit/), and the console output shows xUnit-related errors towards the end.

[xUnit] [INFO] - Starting to record.
[xUnit] [INFO] - Processing BoostTest-1.x (default)
[xUnit] [INFO] - [BoostTest-1.x (default)] - No test report file(s) were found with the pattern 'build/xunit*.xml' relative to '/home/jenkins/workspace/NFD/OS/Ubuntu-14.04-64bit' for the testing framework 'BoostTest-1.x (default)'.  Did you enter a pattern relative to the correct directory?  Did you generate the result report(s) for 'BoostTest-1.x (default)'?
[xUnit] [WARNING] - No test reports found for the metric 'BoostTest' with the resolved pattern 'build/xunit*.xml'.
[xUnit] [INFO] - Skipping the metric tool processing.
[xUnit] [INFO] - There are errors when processing test results.
[xUnit] [INFO] - Skipping tests recording.

This issue affects all Ubuntu slaves. OSX slaves do not seem to be affected.


Related issues 1 (0 open1 closed)

Related to NFD - Bug #3417: Boost 1.60 breaks secondary log output in unit testsClosedAlex Afanasyev01/15/2016

Actions
Actions #1

Updated by Eric Newberry over 8 years ago

  • Status changed from New to In Progress

The current Jenkins scripts run the unit tests with the "--log_sink2" option. However, I could find no documentation of this option and this command produced no output to the reference file when I ran it manually. When I removed the '2', changing the option to "--log_sink" (which is documented), the output appears to be produced correctly.

Actions #2

Updated by Eric Newberry over 8 years ago

It appears that the "--log_format2" option needs to be changed to "--log_format" as well. I'm currently compiling a list of projects affected by this issue.

Actions #3

Updated by Davide Pesavento over 8 years ago

That doesn't explain why OS X slaves are unaffected...

Actions #4

Updated by Davide Pesavento over 8 years ago

--log_format2 and --log_sink2 are custom options, added by us, see https://github.com/named-data/NFD/blob/master/tests/main.cpp

Actions #6

Updated by Eric Newberry over 8 years ago

Davide Pesavento wrote:

Is $XUNIT set in the jenkins environment? (https://github.com/named-data/NFD/blob/master/.jenkins.d/20-tests.sh#L28)

The logging options are only included on the command line if that's the case. It's set in the build shell like so:

XUNIT=1 ./.jenkins

I looked at the help for the unit tests (unit-tests-core --help) and both --log_sink2 and --log_format2 are not listed. As such, I don't believe our customizations are being compiled into the tests on Ubuntu.

@Alex Could you post the help message from one of the OS X nodes for comparison?

Actions #7

Updated by Davide Pesavento over 8 years ago

Eric Newberry wrote:

I looked at the help for the unit tests (unit-tests-core --help) and both --log_sink2 and --log_format2 are not listed.

That's expected I think, only Boost.Test built-in options are shown in --help output. However, note the following behavior:

davide@ndnbox:~/NFD$ ./build/unit-tests-core --log_sink2
ERROR: the required argument for option '--log_sink2' is missing
  --log_format2 arg     Type of second log formatter: HRF or XML
  --log_sink2 arg (=-)  Second log sink, - for stdout

Test setup error: std::runtime_error: test module initialization failed

davide@ndnbox:~/NFD$ ./build/unit-tests-core -- --log_sink2
Running 63 test cases...
[...]
Actions #8

Updated by Davide Pesavento over 8 years ago

  • Assignee changed from Eric Newberry to Alex Afanasyev

Seems to be a regression introduced in NFD commit 3152abac673382b13c39e7df46d830ec9abfc3e0. Other projects are very likely affected as well. Reassigning to Alex, author of the commit.

Actions #9

Updated by Davide Pesavento over 8 years ago

  • Related to Bug #3417: Boost 1.60 breaks secondary log output in unit tests added
Actions #10

Updated by Alex Afanasyev over 8 years ago

Huh.. For Boost < 1.60, we need to move -- in the command line :(. How did I not notice it in the original issue.

Should we make a fix as a separate commit(s)?

Actions #11

Updated by Davide Pesavento about 8 years ago

Alex Afanasyev wrote:

Huh.. For Boost < 1.60, we need to move -- in the command line :(. How did I not notice it in the original issue.

meh, do we need yet another conditional in the CI script? is there no way to have a command line compatible with all versions?

Should we make a fix as a separate commit(s)?

I'm confused, separate from what?

Actions #12

Updated by Alex Afanasyev about 8 years ago

Huh.. For Boost < 1.60, we need to move -- in the command line :(. How did I not notice it in the original issue.

meh, do we need yet another conditional in the CI script? is there no way to have a command line compatible with all versions?

I thought -- would be a compatible syntax. But from what I see, the old version, simply ignores everything after -- and the new version will complain for any non-standard parameter before --

Should we make a fix as a separate commit(s)?

I'm confused, separate from what?

Separate from https://gerrit.named-data.net/#/c/3079/

Actions #14

Updated by Alex Afanasyev about 8 years ago

So, the good part of the new change is that we would not need any custom code to have multiple output types. The bad part that we will need 3 customized ways to run the unit tests... One for old boost, one for boost 1.60-1.61, and finally for boost >1.62.

I'll try to make a try for this.

Actions #15

Updated by Alex Afanasyev about 8 years ago

  • Status changed from In Progress to Code review

https://gerrit.named-data.net/#/c/3128/, but it needs a test with boost 1.62 as soon as it is released (should be very soon) or at least with release candidates.

Actions #16

Updated by Davide Pesavento about 8 years ago

  • Target version set to v0.5
  • % Done changed from 0 to 70
Actions #17

Updated by Davide Pesavento about 8 years ago

  • Status changed from Code review to Closed
  • % Done changed from 70 to 100
Actions

Also available in: Atom PDF