NFD

And for anyone that wants to monitor this on some nodes, here is the link
to the cacti monitoring graphs:

http://ndndemo.arl.wustl.edu/cacti/index.php

You can log in as guest with password ndnTest

Once you are logged in you can click on the tab in the upper right that looks like a waveform.
The dropdown menus at the top then allow you to select specific sets of graphs based
on type or node. For example, if you select "NFD Counters" from the Template menu
you will get the graphs show the NameTree and PIT counters for all nodes.
If you click on one of the graphs it will show you that nodes data over different sample periods.

One interesting thing I notice is that some nodes show the NameTree counter increasing and some don't.

ARIZONA, REMAP, UCI and UCLA appear relatively flat.

CAIDA, CSU, MEMPHIS and WU appear to grow.

I am currently only monitoring 8 nodes. I do plan to add more when I get time.

In case one wants to play with the debugger, I've created Vagrant environment with all the tools I've used: https://github.com/cawka/nfd-debug-environment

I don't see obvious error in Pit::findOrInsert and Pit::erase implementation.

I try to reproduce this bug in a VM, but nfd-status -v somehow isn't working; I'm investigating.

I think I've found the problem.

std::pair<shared_ptr<pit::Entry>, bool>
Pit::findOrInsert(const Interest& interest, bool allowInsert)
{
  (omit)
  shared_ptr<name_tree::Entry> nte = m_nameTree.lookup(isEndWithDigest ? name.getPrefix(-1) : name);

  (omit)

  if (!allowInsert) {
    return {nullptr, true};
  }

  auto entry = make_shared<pit::Entry>(interest);
  (omit)
  return {entry, true};
}

When this function is invoked with allowInsert=false, and the name tree entry doesn't already exist, return {nullptr, true}; is executed but the name tree entry created during m_nameTree.lookup is leaked.

I'm currently waiting for @Weiwei to complete #3608 or reassign it to me, to avoid conflicts in pit.cpp.
After that, I'll work on a patch.

Can we have a similar problem from other tables, e.g., Measurements?

Reply to note-10:

There's no evidence indicating so from note-4.

Which commit of NFD this function (Pit::findOrInsert) was introduced in pit.cpp for the first time? I do not find this function in NFD under ndnSIM. What other code is not merged into NFD in ndnSIM?

https://github.com/named-data-ndnSIM/NFD/blob/bd8eea71137bc382fe1fb8225334b926a5484527/daemon/table/pit.cpp

Reply to note-13:

The fix for PIT leaking NameTree entry is commit:28797798044d0b80c8b883a821645632db9552e5 (it has been merged), as referenced in this Redmine issue.

You may try ndn-cxx ndn-cxx:commit:4f512fb24e8105e2b394ae7528f6236905b50ab4 (it has been merged) and NFD http://gerrit.named-data.net/#/c/2864/2.
This would include fixed for both #3618 and #3619.
We are interested to know whether there's any memory leak elsewhere.

Agreed. The Name Tree fix you are talking about is in master NFD branch.

What I am saying that the same is not available in ndnSIM's NFD fork. In fact the "pit.cpp" file is about 12 commits behind the master NFD. How do we go about this? I have already taken ndn-cxx ndn-cxx:4f512fb24e8105e2b394ae7528f6236905b50ab4 (it has been merged) and NFD http://gerrit.named-data.net/#/c/2864/2. Not sure how to take in 28797798044d0b80c8b883a821645632db9552e5.

Reply to note-16:

You'll need to run your scenario without ndnSIM (eg. in Mini-NDN).

Alternatively, wait for the next ndnSIM release that incorporates NFD 0.4.0 or above, then apply all commits after that.

I can not use Mini-NDN for my tests.
By when the next ndnSIM release with NFD 0.4.0 will be released?

Reply to note-18:

I have posted this question to ndnSIM mailing list.
http://www.lists.cs.ucla.edu/pipermail/ndnsim/2016-May/002750.html

Please wait for an answer from ndnSIM developers.