Project

General

Profile

Actions

Task #1215

closed

Investigate how and set up code coverage reporting

Added by Alex Afanasyev about 10 years ago. Updated about 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Build
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:

Description

We need to check code coverage for our test cases. This probably can be done within Jenkins or using Coverall project


Files

Snip20140217_1.png (451 KB) Snip20140217_1.png Yi Huang, 02/17/2014 02:37 PM

Related issues 1 (0 open1 closed)

Blocked by NFD - Task #1164: Setup Jenkins to perform basic integrated tests and connect it to Gerrit Code ReviewClosedYi Huang01/29/2014

Actions
Actions #1

Updated by Junxiao Shi about 10 years ago

  • Category changed from Tools to Build
Actions #2

Updated by Yi Huang about 10 years ago

  • Status changed from New to In Progress
Actions #3

Updated by Yi Huang about 10 years ago

  • % Done changed from 0 to 40

I spent couple of hours on running unit tests for ndn-cpp-dev on my laptop but it is still failing. Then I switched to ndnd-tlv (unit tests for ndnd-tlv works for me) since fixing the failing unit test is not part of this task and all I need is a project with test that can be used to do experiment with code coverage tools.

I was trying to use "COVERALLS FOR C/C++". However, gcov does not seem to work well with waf. There is another solution described online which uses gcovr and Jenkins' Cobertura Plugin. Trying this now.

Actions #5

Updated by Yi Huang about 10 years ago

To have code coverage checking, the code has to be compiled to generate some temporary files (for gcov, two kinds of file, ".gcno" and ".gcda", are generated) that indicates the code structure and the count of execution of each line.

I followed the instruction from online to run gcovr. However, gcovr shows that everything has a code coverage of 0% (It is not detecting anything even after I run unit tests).

For some reason our code is not generating any ".gcda" files after running unit test when compile with these flags: "-fprofile-arcs -ftest-coverage -fPIC".

Expected behavior:

I have also written a very simple c++ program just like the examples at http://gcovr.com/guide.html. The behavior is as follows:

  1. When I compile with "-fprofile-arcs -ftest-coverage -fPIC", I can see ".gcno" file that is created but no ".gcda" file there.

  2. Then I run the executable produced by g++. After that, the ".gcda" files are created.

  3. After that, I run "gcovr -r .", and I am able to see the output that makes sense.

Actions #6

Updated by Junxiao Shi about 10 years ago

Please submit your change of wscript and other files to Gerrit, so that we can see what's the problem.

Actions #7

Updated by Alex Afanasyev about 10 years ago

Yi and I was playing around yesterday and I suspect that clang is not working well with gcc-style code coverage. Have you tried it on Linux?

Actions #8

Updated by Yi Huang about 10 years ago

Yes I created a all new VM on maestro. Things from my last post are all from that VM (I tested ndn-cpp-dev and did the experiment of simple code there).

To submit wscript, do I commit wscript only and push to origin HEAD:refs/for/new-dev ?

Actions #9

Updated by Alex Afanasyev about 10 years ago

Before submitting, do the following change

Add in the options section:

opt.add_option('--with-coverage',action='store_true',default=False,dest='with_coverage',
               help='''Set compiler flags for gcc to enable code coverage information''')

In configure section:

if conf.options.with_coverage:
    conf.check_cxx(cxxflags=['-fprofile-arcs', '-ftest-coverage', '-fPIC'], uselib_store='GCOV', mandatory=True)

And in the build section, add GCOV to "use" parameter for necessary items.


For ndn-cpp-dev we are using master branch, so submit there. You can also, like Hila, add the topic. I believe it is done like git push origin HEAD:refs/for/master/some_topic_here

Actions #10

Updated by Yi Huang about 10 years ago

I'm not sure how to 'in the build section, add GCOV to "use" parameter for necessary items.'

Do I need to discard the changes I made to 'conf.add_supported_cxxflags' since we are now using the option thing?

Actions #11

Updated by Alex Afanasyev about 10 years ago

Assuming you're modifying ndn-cpp-dev's wscript:

libndn_cpp = bld (
    features=['cxx', 'cxxstlib'], # 'cxxshlib',
    # vnum = "0.3.0",
    target="ndn-cpp-dev",
    name = "ndn-cpp-dev",
    source = bld.path.ant_glob('src/**/*.cpp',
                               excl = ['src/**/*-osx.cpp', 'src/**/*-sqlite3.cpp']),
    use = 'BOOST OPENSSL LOG4CXX CRYPTOPP SQLITE3 RT PIC PTHREAD GCOV',
    includes = ". src",
    export_includes = "src",
    install_path = '${LIBDIR}',
    )

And yes, discard add_supported_flags, since it has slightly different purpose (will just add flags if supported and not going to force/fail when flags are not working, which is desirable in this case).

Actions #12

Updated by Yi Huang about 10 years ago

Cobertura plugin is now work very well with our new configuration option "--with-coverage".

Before start, install Cobertura plugin from our Jenkins website -> Manage Jenkins -> Manage Plugins.

Install Cobertura on our ubuntu slave:

sudo apt-get update

sudo apt-get install cobertura

The changes in Jenkins build configuration should be:

  1. Add "--with-coverage" option when doing "./waf configure". If building requires other packages, this option is needed when building other packages.

  2. At the end of the building script, add one line of command to generate the cobertura friendly xml report. (gcovr -r . --object-directory=build/ -x -o coverage.xml)

  3. In Jenkins project configuration, after the "Build" section, there is a "Post-build Actions" section. Click on "Add post-build action", and then "Publish Cobertura Coverage Report". Then set "Cobertura xml report pattern" with "coverage.xml".

A screenshot indicating how the Jenkins project page with code coverage looks like is attached.

Actions #13

Updated by Junxiao Shi about 10 years ago

What's the intended outcome of this task?

If CodeCoverage document is sufficient, this task should be Closed.

If Jenkins installation is desired, work should continue.

Actions #14

Updated by Junxiao Shi about 10 years ago

  • Status changed from In Progress to Closed
  • % Done changed from 90 to 100

Task is closed as confirmed by @Alex.

Actions

Also available in: Atom PDF