NFD » ndn-cxx

Would be nice to have scatter/gather on the receive side too. I was trying to implement it inside StreamFace today, using a boost::circular_buffer, but I got stuck because Block constructor seems to require a single continuous memory buffer. It shouldn't be hard to do and it could save ourselves at least one memory copy operation in the receive path.

It could be nice, but I don't clearly see real value. The copy operation in the receive is really an artifact that I currently have and can be (partially) easily avoided even without circular buffer: for each new read, just allocate maximum packet size buffer and read, and then create Block from the buffer directly (or multiple blocks from the same buffer, if it happened to be the case).

In any case, I checked and it seems that circular_buffer is only in boost 1.55...

for each new read, just allocate maximum packet size buffer and read, and then create Block from the buffer directly (or multiple blocks from the same buffer, if it happened to be the case).

This solution will not work.

Suppose maximum block size is 8 octets, and a letter indicates the beginning of a packet:

1. Incoming stream is A1234B123C12
2. First read operation receives A1234B12, and delivers block A1234.
3. Second read operation receives 3C12. It must combine B12 left over from first read operation with 3 to deliver a continuous block.

There is a trick to avoid copy and receive exact blocks:

1. Read 10 octets.
   This is guaranteed to contain INTEREST-TYPE | DATA-TYPE and TLV-LENGTH.
   This is also guaranteed not to contain more than one block, because the smallest possible block is .... which is 10 octets.
2. Based on TLV-LENGTH, allocate a buffer enough for the whole block including the first 10 octets, write the first 10 octets into the buffer, and receive subsequent octets.

This trick trades an extra read(2) syscall for a memory copy operation of a whole block. Benchmark is needed to compare the overhead.

I actually don't want to bother optimizing stream-based faces... Personally, I would prioritize datagram-based faces and optimize them, while having stream-based (TCP) just work.

Alex Afanasyev wrote:

In any case, I checked and it seems that circular_buffer is only in boost 1.55...

No, it first appeared in boost 1.35
http://www.boost.org/doc/libs/1_55_0/doc/html/circular_buffer/release.html

oops. I was just changing 1_55_0 to different versions and if the page not there, I assumed it wasn't in the library. In that case, we can investigate this optimization, which in part should be in ndn-cpp-dev (different Block implementation).