Project

General

Profile

Actions

Task #1305

closed

NameTree optimization

Added by Junxiao Shi over 10 years ago. Updated over 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Tables
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
6.00 h

Description

Refine and optimize NameTree implementation:

  • implement better hash function for ndn::Name
  • investigate boost::unordered_map
  • implement shrinking of hashtable nBuckets when utilization is low

Related issues 1 (0 open1 closed)

Follows NFD - Task #1187: NameTreeClosedHaowei Yuan

Actions
Actions #1

Updated by Haowei Yuan over 10 years ago

  • Status changed from New to In Progress
Actions #2

Updated by Haowei Yuan over 10 years ago

  • % Done changed from 0 to 40

Optimization 1: Hash Function

The new hash functions takes a second parameter, which specifies how many components should be used for calculating the hash value, this way, functions like Longest Prefix Matching won't need to make n copies of the name.

Actions #3

Updated by Haowei Yuan over 10 years ago

I am trying to use city hash functionsfor the optimized hash function, this would require adding city.hpp and city.cpp to the repository.

  • At the current stage of the project, shall I place these two files under the /table directory?
  • I tried to put these two files under namespace nfd, but there were some errors. Would it be OK if these two files are not under nfd?
Actions #4

Updated by Junxiao Shi over 10 years ago

city-hash and other third-party code can be placed in core/, and can be kept in its original namespace.

They are also exempt from CodeStyle except that using namespace is not allowed in header files.

Please make sure the code is usable under GPL3.

Actions #5

Updated by Haowei Yuan over 10 years ago

I see. CityHash is using MIT license, which is GPL compatible.

Actions #6

Updated by Junxiao Shi over 10 years ago

  • Status changed from In Progress to Code review
  • % Done changed from 40 to 80
Actions #7

Updated by Junxiao Shi over 10 years ago

  • Status changed from Code review to Closed
  • % Done changed from 80 to 100
Actions #8

Updated by Alex Afanasyev over 10 years ago

What about the second item in the description? Has this been investigated or we abandoning it?

Actions #9

Updated by Haowei Yuan over 10 years ago

Alex Afanasyev wrote:

What about the second item in the description? Has this been investigated or we abandoning it?

I haven't looked into using boost::unordered_map. The original idea was using unordered_map as an optimization, although it was not clear if unordered_map can be used efficiently. I think it is beneficial to look into this problem, so maybe we could create another task, and (probably) this will not be included for the first release.

Actions #10

Updated by Alex Afanasyev over 10 years ago

I'm ok with it. Just wanted to double check.

Technically, just boost::unordered_map may have limitation, but there is also boost::intrusive::unordered_map, that may be a better option, given that we store the same item in multiple places (do we?). In any case, this is a separate issue.

Actions #11

Updated by Haowei Yuan over 10 years ago

Alex Afanasyev wrote:

I'm ok with it. Just wanted to double check.

Technically, just boost::unordered_map may have limitation, but there is also boost::intrusive::unordered_map, that may be a better option, given that we store the same item in multiple places (do we?). In any case, this is a separate issue.

I don't think we store the same item in multiple places. Do you have an example? Some information is stored "multiple times", for example, if we have prefixes /A and /A/B, then we have to store /A and /A/B in two hash buckets (ideally, we need to store only /A and /B, where /B is a suffix of /A). This behavior will not be changed by using boost hash table implementation.

Actions

Also available in: Atom PDF