Task #1215
closedInvestigate how and set up code coverage reporting
Description
We need to check code coverage for our test cases. This probably can be done within Jenkins or using Coverall project
Files
Updated by Yi Huang over 10 years ago
- % Done changed from 0 to 40
I spent couple of hours on running unit tests for ndn-cpp-dev on my laptop but it is still failing. Then I switched to ndnd-tlv (unit tests for ndnd-tlv works for me) since fixing the failing unit test is not part of this task and all I need is a project with test that can be used to do experiment with code coverage tools.
I was trying to use "COVERALLS FOR C/C++". However, gcov does not seem to work well with waf. There is another solution described online which uses gcovr and Jenkins' Cobertura Plugin. Trying this now.
Updated by Alex Afanasyev over 10 years ago
Updated by Yi Huang over 10 years ago
To have code coverage checking, the code has to be compiled to generate some temporary files (for gcov, two kinds of file, ".gcno" and ".gcda", are generated) that indicates the code structure and the count of execution of each line.
I followed the instruction from online to run gcovr. However, gcovr shows that everything has a code coverage of 0% (It is not detecting anything even after I run unit tests).
For some reason our code is not generating any ".gcda" files after running unit test when compile with these flags: "-fprofile-arcs -ftest-coverage -fPIC".
Expected behavior:
I have also written a very simple c++ program just like the examples at http://gcovr.com/guide.html. The behavior is as follows:
When I compile with "-fprofile-arcs -ftest-coverage -fPIC", I can see ".gcno" file that is created but no ".gcda" file there.
Then I run the executable produced by g++. After that, the ".gcda" files are created.
After that, I run "gcovr -r .", and I am able to see the output that makes sense.
Updated by Junxiao Shi over 10 years ago
Please submit your change of wscript
and other files to Gerrit, so that we can see what's the problem.
Updated by Alex Afanasyev over 10 years ago
Yi and I was playing around yesterday and I suspect that clang is not working well with gcc-style code coverage. Have you tried it on Linux?
Updated by Yi Huang over 10 years ago
Yes I created a all new VM on maestro. Things from my last post are all from that VM (I tested ndn-cpp-dev and did the experiment of simple code there).
To submit wscript, do I commit wscript only and push to origin HEAD:refs/for/new-dev ?
Updated by Alex Afanasyev over 10 years ago
Before submitting, do the following change
Add in the options section:
opt.add_option('--with-coverage',action='store_true',default=False,dest='with_coverage',
help='''Set compiler flags for gcc to enable code coverage information''')
In configure section:
if conf.options.with_coverage:
conf.check_cxx(cxxflags=['-fprofile-arcs', '-ftest-coverage', '-fPIC'], uselib_store='GCOV', mandatory=True)
And in the build section, add GCOV to "use" parameter for necessary items.
For ndn-cpp-dev we are using master branch, so submit there. You can also, like Hila, add the topic. I believe it is done like git push origin HEAD:refs/for/master/some_topic_here
Updated by Yi Huang over 10 years ago
I'm not sure how to 'in the build section, add GCOV to "use" parameter for necessary items.'
Do I need to discard the changes I made to 'conf.add_supported_cxxflags
' since we are now using the option thing?
Updated by Alex Afanasyev over 10 years ago
Assuming you're modifying ndn-cpp-dev's wscript:
libndn_cpp = bld (
features=['cxx', 'cxxstlib'], # 'cxxshlib',
# vnum = "0.3.0",
target="ndn-cpp-dev",
name = "ndn-cpp-dev",
source = bld.path.ant_glob('src/**/*.cpp',
excl = ['src/**/*-osx.cpp', 'src/**/*-sqlite3.cpp']),
use = 'BOOST OPENSSL LOG4CXX CRYPTOPP SQLITE3 RT PIC PTHREAD GCOV',
includes = ". src",
export_includes = "src",
install_path = '${LIBDIR}',
)
And yes, discard add_supported_flags, since it has slightly different purpose (will just add flags if supported and not going to force/fail when flags are not working, which is desirable in this case).
Updated by Yi Huang over 10 years ago
- File Snip20140217_1.png Snip20140217_1.png added
- % Done changed from 40 to 90
Cobertura plugin is now work very well with our new configuration option "--with-coverage".
Before start, install Cobertura plugin from our Jenkins website -> Manage Jenkins -> Manage Plugins.
Install Cobertura on our ubuntu slave:
sudo apt-get update
sudo apt-get install cobertura
The changes in Jenkins build configuration should be:
Add "--with-coverage" option when doing "
./waf configure
". If building requires other packages, this option is needed when building other packages.At the end of the building script, add one line of command to generate the cobertura friendly xml report. (
gcovr -r . --object-directory=build/ -x -o coverage.xml
)In Jenkins project configuration, after the "Build" section, there is a "Post-build Actions" section. Click on "Add post-build action", and then "Publish Cobertura Coverage Report". Then set "Cobertura xml report pattern" with "coverage.xml".
A screenshot indicating how the Jenkins project page with code coverage looks like is attached.
Updated by Junxiao Shi over 10 years ago
What's the intended outcome of this task?
If CodeCoverage document is sufficient, this task should be Closed.
If Jenkins installation is desired, work should continue.
Updated by Junxiao Shi over 10 years ago
- Status changed from In Progress to Closed
- % Done changed from 90 to 100
Task is closed as confirmed by @Alex.