Project

General

Profile

Task #2497

Naming convention for test cases

Added by Alex Afanasyev over 4 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Category:
Docs
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:

Description

I would like to propose a change in the naming convention for test cases. I really don't like that in NFD test cases called exactly like tested module. Alternative way was used in ndn-cxx (prepending test- prefix), but I don't like that either and it has not been consistently used. The main reason I don't like it because given the filename it is unclear that it is a test case, not implementation.

Recently I saw that in some projects the following naming convention is used: <tested-module>.t.cpp. That is, we still use name of what we testing, just appending .t suffix.

UnitTesting wiki page should be updated after a decision is made.

History

#1 Updated by Davide Pesavento over 4 years ago

It seems you're talking about the file name here. What about the actual test suite name as passed to boost macros?

#2 Updated by Davide Pesavento over 4 years ago

  • Category changed from Docs to IntegTests

#3 Updated by Alex Afanasyev over 4 years ago

For the test suite name, we have been consistently using <Folder><Module>. I would like to keep this part as is (.t suffix should not be part of the name). For example, for src/encoding/encoder.hpp|cpp module, there should be tests/.../encoding/encoder.t.cpp and suite name EncodingEncoder.

Alternatively, we could do some nesting with suite names. In other words, for every folder, we create a separate test suite. For the same example, there will be suite name Encoding with sub-suite Encoder. Boost is very happy with suites spreaded across multiple files (they are namespaces).


One more thing that I started to add in test cases is enclosing test suites into additional "tests" namespace. Though this doesn't relate to naming conventions.

#4 Updated by Junxiao Shi over 4 years ago

  • Description updated (diff)
  • Category changed from IntegTests to Docs

Category "Tests" refers to "Integration Tests". This issue is unrelated to integ.

For the test suite name, we have been consistently using <Folder><Module>. I would like to keep this part as is (.t suffix should not be part of the name). For example, for src/encoding/encoder.hpp|cpp module, there should be tests/.../encoding/encoder.t.cpp and suite name EncodingEncoder.

One problem with <Folder><Module> appears in ndn-cxx test suites for types directly under src/.

This convention requires test suite for ndn::Interest to be named Interest, which would generate ndn::tests::Interest type.
Testing code under this test suite cannot use Interest to refer to ndn::Interest type because it refers to ndn::tests::Interest type.

I suggest using 'Test'<Module> such as TestInterest for this special case.

Another problem is with the namespace.
Current guidelines require all testing code to be under nfd::tests namespace.

This convention requires test suite for nfd::fw::RetxSuppressionFixed to appear in nfd::tests namespace or its sub-namespace nfd::tests::fw, which means testing code must write fw::RetxSuppressionFixed to refer to the tested type.
Placing into nfd::fw::tests is also bad because testing helpers such as nfd::tests::makeInterest would need a full name.

I suggest placing testing code into nfd::fw::tests namespace to make tested code available, and put using namespace nfd::tests on the top (as a exception to the general guideline of forbidding using namespace) to make testing helpers available.

#5 Updated by Alex Afanasyev over 4 years ago

I have no objections. Just to summarize:

  • for src/module-name.hpp

    • Filename for tests: tests/.../module-name.t.cpp (... is unit-tests, integration-tests, or something else project-specific)
    • Name of the suite TestModuleName
    • Inside the same namespace as src/module-name.hpp plus inside additional tests namespace. If needed, parent tests namespaces can be imported using using namespace xxx::tests directive
  • for src/folder/module-name.hpp

    • Filename for tests: tests/.../folder/module-name.t.cpp
    • Name of the suite FolderModuleName
    • Inside the same namespace as src/folder/module-name.hpp plus inside additional tests namespace. If needed, parent tests namespaces can be imported using using namespace xxx::tests directive

#6 Updated by Junxiao Shi over 4 years ago

I agree with note-5. This Task, once assigned, should update UnitTesting wiki page.

#7 Updated by Davide Pesavento over 4 years ago

Are we going to mass rename the test files in one go or do it incrementally?

#8 Updated by Alex Afanasyev over 4 years ago

Which way you prefer?

I would do mass update in ndn-cxx, as naming there is a little messy now. For nfd we could do incremental or mass update, I have no strong opinion on that.

#9 Updated by Davide Pesavento over 4 years ago

I don't mind either way. If you just do a mechanical mass rename with git (git mv), it should be straightforward to rebase existing commits as git should be able to resolve the conflicts automatically.

#10 Updated by Junxiao Shi over 4 years ago

  • Assignee set to Junxiao Shi
  • Target version set to v0.4

I'll update UnitTesting wiki page with decision in note-5.

I won't do code updates.

#11 Updated by Alex Afanasyev over 4 years ago

I can take over code updates

#12 Updated by Junxiao Shi over 4 years ago

  • Assignee deleted (Junxiao Shi)

UnitTesting page is updated to rev5.

#13 Updated by Spyros Mastorakis over 4 years ago

  • Status changed from New to In Progress
  • Assignee set to Spyros Mastorakis

#14 Updated by Spyros Mastorakis over 4 years ago

  • Status changed from In Progress to Code review
  • % Done changed from 0 to 90

#15 Updated by Spyros Mastorakis over 4 years ago

  • Status changed from Code review to Closed
  • % Done changed from 90 to 100

#16 Updated by Alex Afanasyev over 4 years ago

  • Status changed from Closed to Code review

Not all code is merged yet :)

#17 Updated by Spyros Mastorakis over 4 years ago

  • Status changed from Code review to Closed

#18 Updated by Junxiao Shi about 4 years ago

In ndn-cxx:commit:bae21d51840dbcee046ed67a7337db68d823971e, test suite for ndn::security::transform::Base64Decode is being nested like:

BOOST_AUTO_TEST_SUITE(Security)
BOOST_AUTO_TEST_SUITE(TransformBase64Decode)

The idea could be correct, but this violates the convention set here.

We need to re-discuss the convention.

#19 Updated by Junxiao Shi about 4 years ago

20150901 conference call decides to change the convention to UnitTesting rev6.

Existing test suites may retain their current names, but new test suites should follow the new convention.

Also available in: Atom PDF