Project

General

Profile

Actions

Feature #3168

closed

UdpTransport

Added by Junxiao Shi about 9 years ago. Updated about 9 years ago.

Status:
Closed
Priority:
High
Assignee:
Category:
Faces
Target version:
Start date:
Due date:
% Done:

100%

Estimated time:
3.00 h

Description

Implement UdpTransport for use with LinkService.

The UdpTransport is a subclass of Transport that communicates with either one remote forwarder or a multicast group via a UDP socket.

  • UdpTransport is always non-local.
  • UdpTransport can be on-demand, persistent, or permanent.

After implementing UdpTransport:

  • change UdpChannel to initialize an LpFace(GenericLinkService+UdpTransport) in place of UdpFace
  • delete old UdpFace and MulticastUdpFace

Related issues 5 (0 open5 closed)

Blocked by NFD - Task #3088: Refactor Face as LinkService+TransportClosedEric Newberry

Actions
Blocks NFD - Task #3172: Refactor Face: completionClosedJunxiao Shi

Actions
Blocked by NFD - Feature #3179: EndpointIdClosedJunxiao Shi

Actions
Blocks NFD - Bug #3262: MulticastUdpTransport does not set remoteEndpointClosedDavide Pesavento

Actions
Blocks NFD - Task #3307: UnicastUdpTransport: test suite improvementClosedEric Newberry

Actions
Actions #1

Updated by Junxiao Shi about 9 years ago

  • Blocked by Task #3088: Refactor Face as LinkService+Transport added
Actions #2

Updated by Junxiao Shi about 9 years ago

  • Description updated (diff)

Changes for this issue should be uploaded to feature-lp branch.

Most code can be adapted from old UdpFace and MulticastUdpFace.

Initialization logic is partially designed in #3088 note-20.

The assignee may decide whether to have a separate MulticastUdpTransport or to implement both unicast and multicast in the same UdpTransport class.

The assignee may decide whether to have a DatagramTransport template that is shared between UdpTransport and future datagram-based transports such as UnixSeqPacketTransport (#1672), or to implement them separately.

Actions #3

Updated by Junxiao Shi about 9 years ago

Actions #4

Updated by Junxiao Shi about 9 years ago

  • Description updated (diff)
Actions #5

Updated by Junxiao Shi about 9 years ago

  • Assignee set to Yukai Tu
Actions #6

Updated by Yukai Tu about 9 years ago

Should I wait for Eric finish his task?

Actions #7

Updated by Davide Pesavento about 9 years ago

Actions #8

Updated by Davide Pesavento about 9 years ago

Junxiao Shi wrote:

The assignee may decide whether to have a DatagramTransport template that is shared between UdpTransport and future datagram-based transports such as UnixSeqPacketTransport (#1672), or to implement them separately.

UnixSeqPacketTransport is not datagram-based (as in SOCK_DGRAM-based). It is based on the SOCK_SEQPACKET socket type. In any case I wouldn't worry about sharing code between the two, as it is not clear yet whether a UnixSeqPacketTransport will ever be implemented.

Actions #9

Updated by Junxiao Shi about 9 years ago

  • Description updated (diff)
Actions #10

Updated by Junxiao Shi about 9 years ago

  • Blocks Task #3172: Refactor Face: completion added
Actions #11

Updated by Junxiao Shi about 9 years ago

Should I wait for Eric finish his task?

Implementation cannot start until #3088 is merged.

However, you may start reading existing UdpFace code and Transport API.

Actions #12

Updated by Junxiao Shi about 9 years ago

Actions #13

Updated by Yukai Tu about 9 years ago

  • Status changed from New to In Progress
  • Estimated time changed from 3.00 h to 1.00 h
Actions #14

Updated by Yukai Tu about 9 years ago

Should I submit UdpTransport without LinkService and Transport? And should I create a new unit-test for transport, or just modify the original udp.t.cpp?

Thanks!

Actions #15

Updated by Junxiao Shi about 9 years ago

Answer to note-14:

Should I submit UdpTransport without LinkService and Transport?

The commit to upload should have #3088 as the parent commit.

(I have explained how to do this in a conference call with the assignee)

And should I create a new unit-test for transport, or just modify the original udp.t.cpp?

I'm looking through udp.t.cpp and it appears that existing test cases should continue to work, unchanged, with LpFaceWrapper(LpFace(GenericLinkService+UdpTransport)).

I'm unsure about the necessity of new test cases for UdpTransport.

Actions #16

Updated by Yukai Tu about 9 years ago

I may separate MultipleUdpTransport since UdpTransport and MultipleUdpTransport have different doSend(): async_send and async_send_to.

So I will keep datagramTransport. And it will minimize the damage to original process.

Actions #17

Updated by Yukai Tu about 9 years ago

One more question:

Should I put UdpChannal into namespace Face?

Actions #18

Updated by Davide Pesavento about 9 years ago

Yukai Tu wrote:

I may separate MultipleUdpTransport since UdpTransport and MultipleUdpTransport have different doSend(): async_send and async_send_to.

What is MultipleUdp? do you mean Multicast?

The difference between async_send and async_send_to isn't really a difference for UDP, you can use async_send_to in both cases if you want.

Actions #19

Updated by Davide Pesavento about 9 years ago

Yukai Tu wrote:

Should I put UdpChannal into namespace Face?

I think in another ticket I've already argued against excessive namespacing in NFD, so I would disagree with moving the whole faces subsystem into namespace face.

Actions #20

Updated by Yukai Tu about 9 years ago

Davide Pesavento wrote:

The difference between async_send and async_send_to isn't really a difference for UDP, you can use async_send_to in both cases if you want.

Yes, except this difference, it seems that combining two transport is accetable. The original design uses async_send in UdpFace since it can avoid reimplementing dst/port-based dispatch manually(http://redmine.named-data.net/issues/2989, note-37, Alex), I prefer keep these two transport separated.

Actions #21

Updated by Yukai Tu about 9 years ago

  • Estimated time changed from 1.00 h to 3.00 h
Actions #22

Updated by Junxiao Shi about 9 years ago

"dstport-based dispatch" is relevant to receiving only.

async_send_to with the correct remote endpoint should have the same effect as async_send.


No, this issue should not change namespace of UdpChannel or UdpFactory.

Actions #23

Updated by Junxiao Shi about 9 years ago

Yukai and I had a conference call about this issue. Summary:

  • It's necessary to have separate Transports for unicast and multicast, because multicast needs two separate sockets for sending and receiving. The classes should be named UnicastUdpTransport and MulticastUdpTransport, because there's no inheritance relation between these two. Both can inherit from DatagramTransport class template.
  • In UdpChannel::handleNewPeer, face->receiveDatagram(m_inputBuffer, nBytesReceived, error) should be changed to calling a method on UnicastUdpTransport. This requires LpFaceWrapper to have a getter that returns a regular pointer to the underlying LpFace; saving a pointer to the original UnicastUdpTransport or LpFace is ineffective because they have been moved away and there's no guarantee the saved pointer is still valid.
  • Idle detection remains in UnicastUdpTransport. Any incoming packet that can be parsed as valid TLV block indicates "recent usage".
  • If the face should be closed due to idle, UnicastUdpTransport can invoke this->close();.
  • DatagramTransport::doClose should close its socket.
  • MulticastUdpTransport::doClose should invoke DatagramTransport::doClose then close its send socket. This requires Transport::doClose to be changed from private to protected.
Actions #24

Updated by Yukai Tu about 9 years ago

I have compiled the code successfully. When I run unit-tests, there are some unit-tests need Counter.

Actions #25

Updated by Junxiao Shi about 9 years ago

Reply to note-24:

The existing test suite should work just fine with the counters on LpFaceWrapper.

The counters on LpFace Transport aren't incremented, and aren't accessible from existing test suite.

In any case, upload your patchset to Gerrit, so others can have a look.

Actions #26

Updated by Yukai Tu about 9 years ago

In UdpChannel::handleNewPeer, I use LpFaceWrapper::getLpFace()->getTransport, then static_cast it to the UnicastTransport, in which we use receiveDatagram(). The original process will get MutableCounters in datagramFace, but now we don't. That's the point of the failure of the unit-tests.

Any way, I will submit my code for review.

Actions #27

Updated by Junxiao Shi about 9 years ago

Reply to note-26:

I can see the problem comes from: DatagramFace increments link layer byte counter, but DatagramTransport appears not to increment those counters.
DatagramTransport can increment the counter after #3177, but those counters are visible within LpFace and are not exposed in LpFaceWrapper.

I would say, mark these assertions as expected failures for now, until either #3172 or when we find another solution.

Actions #28

Updated by Davide Pesavento about 9 years ago

Junxiao Shi wrote:

"dstport-based dispatch" is relevant to receiving only.

async_send_to with the correct remote endpoint should have the same effect as async_send.

yep, exactly.

Actions #29

Updated by Yukai Tu about 9 years ago

In original process, we sent out the interest.wireEncode(), now we send out lpPacket.wireEncode(). It seems that theirs' bytes are different.

Actions #30

Updated by Junxiao Shi about 9 years ago

Reply to note-29:

They won't be different after #2930 note-18 is implemented.

Actions #31

Updated by Yukai Tu about 9 years ago

Two problems:

  1. In unit-tests, FaceHistory will record the face's failure. In LpFaceWrapper's close method, it dosen't call the fail(), which causes FaceHistory can't record the face failure. I add one line in LpFaceWrapper::close(): this->fail("Face closed"). I think that should be added to #3088.

  2. The same as above, this time is when UdpTransport out of time, it will call Transport::close(). But in Transport, we can't get Face::fail(), so I commend two line in udp.t.cpp, waiting for the future design.

I just submit the UdpTransport on gerrit which can build successfully and trought all unit-tests. I think it can be used by other person who will develop based on it. If anything gose well, I can begin UnixStreamTransport.

Actions #32

Updated by Junxiao Shi about 9 years ago

Reply to note-31:

LpFaceWrapper::close is not the right place to signal (old)Face::onFail. It should hook to LpFace::afterStateChange signal.

I've added the correct solution to commit:191178a0cfb12d9111df782f0e1012cc92dda0df.

Actions #33

Updated by Yukai Tu about 9 years ago

I may need some help about cherry-pick. Since I use command git fetch ssh://xiaoxiaff@gerrit.named-data.net:29418/NFD refs/changes/23/2423/24 && git cherry-pick FETCH_HEAD, after that I solve the conflict manually, I can't amend my commit.

Actions #34

Updated by Alex Afanasyev about 9 years ago

Hi Yukai,

You need to have different order. As a general rule, you don't cherry-pick other commits on top of your commit. You cherry pick or rebase your commit on top of others.

The following commands will have equivalent effect:

# I'm assuming your commit is ab872cda0de6a54c76d2f9a71d6a572ac5fcb568

# if you are not yet checkout your commit
git checkout -b udp-transport ab872cda0de6a54c76d2f9a71d6a572ac5fcb568

# Retrieve existing code, which will be a new base for your commit
git fetch http://gerrit.named-data.net/NFD refs/changes/23/2423/24

# Request interactive rebase on top of just retrieved code
git rebase -i FETCH_HEAD

# In the presented view, select which commits you want to apply, save, and quit
# resolve conflicts using
git mergetool

# Finish up with
git rebase --continue

Same thing, but explicitly with cherry-pick

# Retrieve existing code, which will be a new base for your commit
git fetch http://gerrit.named-data.net/NFD refs/changes/23/2423/24

# Create a new branch for the code based on just fetched code
git checkout -b udp-transport FETCH_HEAD

# Cherry-pick your code
git cherry-pick ab872cda0de6a54c76d2f9a71d6a572ac5fcb568

# Resolve conflicts
git mergetool

# Finish up
git cherry-pick --continue

#or just 
git commit
Actions #35

Updated by Yukai Tu about 9 years ago

Hi Alex:

Thanks for your help. I had found my commit id.

Actions #36

Updated by Yukai Tu about 9 years ago

When I finished all these steps, I commit my code. But it said that Eric's commit's author email doesn't match my email account.

Actions #37

Updated by Yukai Tu about 9 years ago

  • Status changed from In Progress to Code review
Actions #38

Updated by Junxiao Shi about 9 years ago

  • Status changed from Code review to Closed
  • % Done changed from 0 to 100
Actions #39

Updated by Junxiao Shi about 9 years ago

  • Blocks Bug #3262: MulticastUdpTransport does not set remoteEndpoint added
Actions #40

Updated by Junxiao Shi about 9 years ago

  • Blocks Task #3307: UnicastUdpTransport: test suite improvement added
Actions

Also available in: Atom PDF