Task #2538
Updated by Alex Afanasyev about 9 years ago
I address this issue as a task, because I do not really know whether there are indeed such issues and, if so, whether they are related to ndnSIM or NFD. Potential memory leaks were reported by Christian Kreuzberger in ndnSIM mailing list. Part of the original email: using the latest version of ndnSIM and ndn-cxx, I’ve noticed a rather high memory usage (several gigabytes, with content store set to 1 packet) in simulations involving plenty of interests and data packets. So I’ve compiled ndnSIM in debug mode and started the ndn-simple example (from scratch directory) with valgrind active, here is the summary of the output. ./waf --command-template="valgrind --leak-check=full --show-reachable=yes %s" --run ndn-simple ==9678== 1,738,712 (408 direct, 1,738,304 indirect) bytes in 3 blocks are definitely lost in loss record 448 of 448 ==9678== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==9678== by 0x5AD027F: ns3::Ptr<ns3::Node> ns3::CreateObject<ns3::Node>() (object.h:423) ==9678== by 0xD9C8CFA: ns3::NodeContainer::Create(unsigned int) (node-container.cc:137) ==9678== by 0x41C856: ns3::main(int, char**) (ndn-simple.cc:63) ==9678== by 0x41D2D7: main (ndn-simple.cc:107) ==9678== ==9678== LEAK SUMMARY: ==9678== definitely lost: 408 bytes in 3 blocks ==9678== indirectly lost: 1,738,304 bytes in 8,257 blocks ==9678== possibly lost: 98 bytes in 3 blocks ==9678== still reachable: 656 bytes in 33 blocks ==9678== suppressed: 0 bytes in 0 blocks At first, this looks like that the memory lost is only when creating a node, which is not good, but would be “okay” for simulations. But when you dig deeper into the generated output of valgrind, you will find that the problem is caused by several allocations about packets, regular expressions, skiplists in content store, etc… I’m not an expert in that field, but I thought it is worth mentioning on the mailing list. Example of the most important ones: ==9806== 238,400 bytes in 200 blocks are indirectly lost in loss record 447 of 448 ==9806== at 0x4C2B0E0: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==9806== by 0x8DE20A2: __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<ndn::Data, std::allocator<ndn::Data>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) (new_allocator.h:104) ==9806== by 0x8DE1D3D: std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<ndn::Data, std::allocator<ndn::Data>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<ndn::Data, std::allocator<ndn::Data>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) (alloc_traits.h:351) ==9806== by 0x8DE18E7: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<ndn::Data, std::allocator<ndn::Data>>(std::_Sp_make_shared_tag, ndn::Data*, std::allocator<ndn::Data> const&) (shared_ptr_base.h:499) ==9806== by 0x8DE138D: std::__shared_ptr<ndn::Data, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<ndn::Data>>(std::_Sp_make_shared_tag, std::allocator<ndn::Data> const&) (shared_ptr_base.h:957) ==9806== by 0x8DE0B6B: std::shared_ptr<ndn::Data>::shared_ptr<std::allocator<ndn::Data>>(std::_Sp_make_shared_tag, std::allocator<ndn::Data> const&) (shared_ptr.h:316) ==9806== by 0x8DDFC5B: std::shared_ptr<ndn::Data> std::allocate_shared<ndn::Data, std::allocator<ndn::Data>>(std::allocator<ndn::Data> const&) (shared_ptr.h:598) ==9806== by 0x8DDF103: std::shared_ptr<ndn::Data> std::make_shared<ndn::Data>() (shared_ptr.h:614) ==9806== by 0x8FD6B9C: ns3::ndn::PacketHeader<ndn::Data>::Deserialize(ns3::Buffer::Iterator) (ndn-header.cpp:120) ==9806== by 0xD912BAA: ns3::Packet::RemoveHeader(ns3::Header&) (packet.cc:290) ==9806== by 0x8FF2F53: std::shared_ptr<ndn::Data const> ns3::ndn::Convert::FromPacket<ndn::Data>(ns3::Ptr<ns3::Packet>) (ndn-ns3.cpp:37) ==9806== by 0x8FF18DD: ns3::ndn::NetDeviceFace::receiveFromNetDevice(ns3::Ptr<ns3::NetDevice>, ns3::Ptr<ns3::Packet const>, unsigned short, ns3::Address const&, ns3::Address const&, ns3::NetDevice::PacketType) (ndn-net-device-face.cpp:130) ==9806== ==9806== 216,879 bytes in 201 blocks are indirectly lost in loss record 446 of 448 ==9806== at 0x4C2B800: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==9806== by 0xD8E7074: ns3::Buffer::Allocate(unsigned int) (buffer.cc:172) ==9806== by 0xD8E6D93: ns3::Buffer::Create(unsigned int) (buffer.cc:141) ==9806== by 0xD8E8666: ns3::Buffer::AddAtStart(unsigned int) (buffer.cc:331) ==9806== by 0xD912A53: ns3::Packet::AddHeader(ns3::Header const&) (packet.cc:278) ==9806== by 0xAD8A64A: ns3::PointToPointNetDevice::AddHeader(ns3::Ptr<ns3::Packet>, unsigned short) (point-to-point-net-device.cc:167) ==9806== by 0xAD8E466: ns3::PointToPointNetDevice::Send(ns3::Ptr<ns3::Packet>, ns3::Address const&, unsigned short) (point-to-point-net-device.cc:502) ==9806== by 0x8FF10E5: ns3::ndn::NetDeviceFace::send(ns3::Ptr<ns3::Packet>) (ndn-net-device-face.cpp:89) ==9806== by 0x8FF150C: ns3::ndn::NetDeviceFace::sendData(ndn::Data const&) (ndn-net-device-face.cpp:111) ==9806== by 0x8DF6F06: nfd::Forwarder::onOutgoingData(ndn::Data const&, nfd::Face&) (forwarder.cpp:377) ==9806== by 0x8DF6455: nfd::Forwarder::onIncomingData(nfd::Face&, ndn::Data const&) (forwarder.cpp:333) ==9806== by 0x8DEDA96: nfd::Forwarder::onData(nfd::Face&, ndn::Data const&) (forwarder.hpp:263) ==9806== The valgrind output is rather long, I suggest you try generating it by yourself. I’m using Ubuntu 14.04, if that matters. I’m not after chasing every single memory leak, though if it involves packets/packet based structures, there might be an issue with ndnSIM or even NFD, that could affect the testbed aswell. In addition, I’ve tested ndn-simple with Frequency set to other values instead of 10, here is the resulting output: http://pastebin.com/0PZw40bk