NFD

@Davide will justify the motivation.

The major challenge is to determine path MTU, so that fragmented LpPacket does not get fragmented again by IP.

Junxiao Shi wrote:

The major challenge is to determine path MTU, so that fragmented LpPacket does not get fragmented again by IP.

NDNLP fragmentation is hop-by-hop, why do we need to know the path MTU?

NDNLP fragmentation is hop-by-hop

Here "hop" means "NDN hop", between two NDN nodes.

why do we need to know the path MTU?

An "NDN hop" may cross multiple IP hops with different MTUs. The lowest MTU among those IP hops is the "UDP path MTU".

Oh ok, I misunderstood your comment, you're talking about the "IP path". Note that UnicastUdpTransport explicitly disables path MTU discovery.

There's no rationale for this task in the description.

Junxiao Shi wrote:

@Davide will justify the motivation.

The major challenge is to determine path MTU, so that fragmented LpPacket does not get fragmented again by IP.

I get the intuitive reasoning (1 fragmentation is better than 2), but is there any evidence that this will actually improve performance?

I could imagine the case where NDN fragmentation is the bottleneck and it doesn't make much difference whether there is additional IP fragmentation or not.

However, one rationale could be Susmit's request to use larger packet sizes over UDP.

This came up again during the 10th Retreat as a solution/workaround for IP fragmentation issues at various NDN testbed nodes (firewalls or middleboxes not under our control dropping fragmented IP packets). As a first approximation, and to simplify the initial implementation, Lixia and others suggested to skip PMTU discovery and assume a reasonable MTU for the path (e.g. 1500 bytes). The MTU value can be configured via nfd.conf or nfdc.

Another obvious reason is to increase adoption and testing of NFD's fragmentation/reassembly code, which I think is not used much at the moment.

Klaus Schneider wrote:

Junxiao Shi wrote:

The major challenge is to determine path MTU, so that fragmented LpPacket does not get fragmented again by IP.

I get the intuitive reasoning (1 fragmentation is better than 2), but is there any evidence that this will actually improve performance?

The goal is not to improve performance. In fact, switching to NDNLP fragmentation will likely decrease performance, since our fragmentation/reassembly code is far less optimized than fragmentation done by IP routers.

However, one rationale could be Susmit's request to use larger packet sizes over UDP.

Yes, that is another reason.

The uses mentioned by Davide above are why I believe we shouldn't enable NDNLPv2 frag/reassembly by default on UDP transports, but rather make it available as an option (disabled by default) through management. We should also allow the parameters of the fragmentation to be set through management.

allowing NDNLPv2 fragmentation and reassembly to be explicitly enabled or disabled on individual faces through management. Through management, one would also be able to set a fragmentation threshold, with LpFragmenter modified to use the minimum of this value (if set) and the MTU (if any) as the fragmentation threshold.

This is over generalization. This issue asks for LP fragmentation on UDP transport. There is no valid use case on any other transport.
The simplest way to implement this is to overwrite UdpTransport’s MTU at construction time. Runtime changes are not currently supported in LpFragmenter, and there’s no use case requiring it.

Are there any transport types where fragmentation and reassembly should not be able to be explicitly enabled or disabled?

Fragmentation makes sense on datagram transport only.

Also, is there any reason to separate enabling fragmentation and enabling reassembly into separate flags for enabling/disabling?

Reassembly should be enabled on all datagram transports. This is already the case and there’s no reason to change it.
Fragmentation should be enabled on all datagram transports. This is already the case and there’s no reason to change it. All you need is allowing to lower MTU on UDP transport, so fragmentation actually happens.

Junxiao Shi wrote:

allowing NDNLPv2 fragmentation and reassembly to be explicitly enabled or disabled on individual faces through management. Through management, one would also be able to set a fragmentation threshold, with LpFragmenter modified to use the minimum of this value (if set) and the MTU (if any) as the fragmentation threshold.

This is over generalization. This issue asks for LP fragmentation on UDP transport. There is no valid use case on any other transport.

See below. This is related (and perhaps supersedes this original issue), so we can discuss it here.

The simplest way to implement this is to overwrite UdpTransport’s MTU at construction time. Runtime changes are not currently supported in LpFragmenter, and there’s no use case requiring it.

Then we would need a way to specify the MTU through management. Regardless, I disagree with doing this because it seems like a dirty hack, since the MTU could be used in other ways by other features.

Are there any transport types where fragmentation and reassembly should not be able to be explicitly enabled or disabled?

Fragmentation makes sense on datagram transport only.

Agreed, but I don't think we should necessarily forbid it from being explicitly enabled on other face types, since it shouldn't do any harm.

Also, is there any reason to separate enabling fragmentation and enabling reassembly into separate flags for enabling/disabling?

Reassembly should be enabled on all datagram transports. This is already the case and there’s no reason to change it.

I don't believe this is the case. From what I can tell, reliability is disabled by default and is only enabled on EthernetTransport.

Fragmentation should be enabled on all datagram transports. This is already the case and there’s no reason to change it. All you need is allowing to lower MTU on UDP transport, so fragmentation actually happens.

Likewise.

We can't simply lower the MTU of a face at the moment because the option to set this isn't available through management.

Additionally, at the retreat, John DeHart said this feature is needed to allows the use of UDP tunnels in scenarios where an intermediate firewall will drop IP fragments. Therefore, we don't need to entirely switch all UDP faces over the LpFragmenter/Reassembler, but rather allow it to be used instead of IP fragmentation if need be.

Fragmentation makes sense on datagram transport only.

Agreed, but I don't think we should necessarily forbid it from being explicitly enabled on other face types, since it shouldn't do any harm.

Fragmentation makes no sense on a stream transport. The harm is over complication and performance penalty.

is there any reason to separate enabling fragmentation and enabling reassembly into separate flags for enabling/disabling?

It seems that they already have separate flags from NFD:commit:4c3e6b89a17c34d642d04fd5b37df447a939b9a6. In this case, there's no reason to change it.

We can't simply lower the MTU of a face at the moment because the option to set this isn't available through management.

Lowering MTU of unicast UDP transport is the feature needed to solve this issue.

Additionally, at the retreat, John DeHart said this feature is needed to allows the use of UDP tunnels in scenarios where an intermediate firewall will drop IP fragments. Therefore, we don't need to entirely switch all UDP faces over the LpFragmenter/Reassembler, but rather allow it to be used instead of IP fragmentation if need be.

All unicast UDP faces should enable fragmentation and reassembly. MTU of unicast UDP transport normally stays at 64K (higher than 8800-octet) so that fragmentation does not occur. nfdc face create command sets a lower MTU where IP fragmentation is unavailable, allowing fragmentation to occur.

Through management, one would also be able to set a fragmentation threshold, with LpFragmenter modified to use the minimum of this value (if set) and the MTU (if any) as the fragmentation threshold.

This would not work. Piggybacking algorithm does not pick up this "fragmentation threshold", and thus other LP features can insert more headers and cause the LpPacket to go over IP link's PMTU.

Junxiao Shi wrote:

Fragmentation makes sense on datagram transport only.

Agreed, but I don't think we should necessarily forbid it from being explicitly enabled on other face types, since it shouldn't do any harm.

Fragmentation makes no sense on a stream transport. The harm is over complication and performance penalty.

Ok. If we add fragmentation and reassembly as face management Flags, we could simply forbid them from being enabled on non-datagram faces.

is there any reason to separate enabling fragmentation and enabling reassembly into separate flags for enabling/disabling?

It seems that they already have separate flags from NFD:commit:4c3e6b89a17c34d642d04fd5b37df447a939b9a6. In this case, there's no reason to change it.

I was referring to management Flags, but I don't disagree with this justification. Could you give a scenario where one would want fragmentation enabled, but not reassembly, or vice versa?

If we add fragmentation and reassembly as face management Flags, we could simply forbid them from being enabled on non-datagram faces.

Unnecessary to forbid in management. Since MTU stays "infinite", fragmentation can never occur.

Could you give a scenario where one would want fragmentation enabled, but not reassembly

There isn't a use case for this.

or vice versa?

A consumer's face may only send small Interests, but expects larger Data.

is there any reason to separate enabling fragmentation and enabling reassembly into separate flags for enabling/disabling?

I can agree with combining them, but this is out of scope of this issue.

Junxiao Shi wrote:

If we add fragmentation and reassembly as face management Flags, we could simply forbid them from being enabled on non-datagram faces.

Unnecessary to forbid in management. Since MTU stays "infinite", fragmentation can never occur.

Could you give a scenario where one would want fragmentation enabled, but not reassembly

There isn't a use case for this.

or vice versa?

A consumer's face may only send small Interests, but expects larger Data.

Would potentially having fragmentation enabled harm anything here?

is there any reason to separate enabling fragmentation and enabling reassembly into separate flags for enabling/disabling?

I can agree with combining them, but this is out of scope of this issue.

I don't believe it is. We haven't agreed to what you said the task should be in note 14 (lowering the MTU instead of allowing reliability to be enabled).

Ok. Given Davide's comments in note 18, perhaps we should add a face management parameter to specify the MTU of the face?

To summarize what I've interpreted of the intended design so far (and adding some more specific details):

• We're going to add an "Mtu" field to the face management ControlParameters for use with face/create.
  • I don't believe this mechanism would work with face/update due to the inability of LpFragmentation to handle runtime changes to MTU.
• The minimum of this user-provided MTU (if any) and the physical MTU of the link will be used to determine the MTU of the Face.
• If the user-provided MTU is omitted, the physical MTU will be used by the face.
• The user-provided MTU will be added as an optional field in FaceStatus to contain the user-provided MTU value (if any).
• Make any necessary changes to nfdc to allow the "Mtu" field to be set when creating a face and obtained when listing faces.

I'd say make it even simpler: whatever MTU is provided by the user via nfdc/management overrides the MTU property of the Transport. No need to maintain two separate MTU values. The default setting (if no mtu is specified on the command line, or the special value "auto" is specified) is still to automatically determine the Transport MTU based on the underlying link.

Davide Pesavento wrote:

I'd say make it even simpler: whatever MTU is provided by the user via nfdc/management overrides the MTU property of the Transport. No need to maintain two separate MTU values. The default setting (if no mtu is specified on the command line, or the special value "auto" is specified) is still to automatically determine the Transport MTU based on the underlying link.

Ok, I'm fine with this.

Notes from the discussion of this issue on the NDN Platform Call (May 21, 2018):

• The minimum of the user-specified MTU in FaceCreateParameters and the physical MTU of the underlying transport will be used to set the MTU of the face.
• If the MTU parameter is omitted in FaceCreateParameters, the MTU value of the face will be set to infinity, making the set MTU be the minimum of infinity and the physical MTU of the underlying transport.
• The specified MTU must be greater than or equal to 64 bytes to ensure fragmentation functions correctly.

I've written the design for this feature on the FaceMgmt wiki page.

In FaceMgmt rev85,

• "static face properties" section describes MTU as:

  This value can only be set with faces/create and cannot be changed on an existing face with faces/update.

  This statement is redundant, because faces/update command does not accept MTU parameter.
  "static face properties" section should only describe "what they are", not "where they can be used".

• faces/create command indicates MTU element is optional. Why not reply with the effective MTU of the face?
  You may define "omitted" as "infinity", but that has to be specified in the protocol.

• "face dataset" section defines:

  Mtu contains the user-provided MTU to override the automatic MTU provided by the transport (in bytes).

  Why not include the effective MTU of the face?

I agree with the changes suggested by Junxiao in note 26 and have updated the FaceMgmt page to reflect them. I will update the ndn-cxx change on Gerrit soon.

I have updated the ndn-cxx change to reflect the changes in r86.

I added the LpFragmentationReassemblyEnabled bit in FaceMgmt r88.

After a discussion with Davide in which we discovered that incoming packets are passed to LpReassembly regardless of whether the feature is enabled, I've decided that the management flag should only affect whether the LpFragmentation feature is enabled. I've updated FaceMgmt in r89 to reflect this and will adjust the ndn-cxx patch shortly.

I added the LpFragmentationEnabled bit in FaceMgmt

I disagree with this change. Fragmentation should remain enabled on every face, but it does not occur if effective MTU is greater than packet size. There’s no need for a management flag.

Junxiao Shi wrote:

I added the LpFragmentationEnabled bit in FaceMgmt

I disagree with this change. Fragmentation should remain enabled on every face, but it does not occur if effective MTU is greater than packet size. There’s no need for a management flag.

On every datagram face or every UDP face?

In it's current state, the flag is meant to control specifically whether LpFragmentation or IP fragmentation is used. I thought this is what we intended the design to be?

On every datagram face or every UDP face?

On every face.

• Stream face: effective MTU is infinity, LP fragmentation does not occur.
• Ethernet face: effective MTU is 1500 (queried from NIC), LP fragmentation may occur.
• UDP and operator wants IP fragmentation: effective MTU (as seen by LinkService) is about 65000, LP fragmentation does not occur.
• UDP and operator wants LP fragmentation: effective MTU is 1400 (overridden on face create), LP fragmentation may occur.

Junxiao Shi wrote:

• UDP and operator wants IP fragmentation: effective MTU (as seen by LinkService) is about 65000, LP fragmentation does not occur.
• UDP and operator wants LP fragmentation: effective MTU is 1400 (overridden on face create), LP fragmentation may occur.

On these two, how would we differentiate whether LP fragmentation may occur or not? Is it just if the MTU is below a threshold (MAX_NDN_PACKET_SIZE)? Or if the MTU value is overridden by the user?

• UDP and operator wants IP fragmentation: effective MTU (as seen by LinkService) is about 65000, LP fragmentation does not occur.
• UDP and operator wants LP fragmentation: effective MTU is 1400 (overridden on face create), LP fragmentation may occur.

On these two, how would we differentiate whether LP fragmentation may occur or not? Is it just if the MTU is below a threshold (MAX_NDN_PACKET_SIZE)? Or if the MTU value is overridden by the user?

You don't differentiate. LP fragmentation occurs when upper layer wants to transmit a packet longer than the effective MTU, regardless of where that MTU comes from.

FaceMgmt r90 has been updated to incorporate Junxiao's suggestion in note 33.

Davide has raised concerns, and I agree with them, that omitted Mtu field in responses have a dual meaning: it is either unlimited Mtu size or Mtu reporting is not supported by the forwarder. I would vote that we dedicate 0 as a special value for "unlimited" designation and update the protocol correspondingly.

Alex Afanasyev wrote:

Davide has raised concerns, and I agree with them, that omitted Mtu field in responses have a dual meaning: it is either unlimited Mtu size or Mtu reporting is not supported by the forwarder. I would vote that we dedicate 0 as a special value for "unlimited" designation and update the protocol correspondingly.

Seems logical to me. I "+1" this proposal.

omitted Mtu field in responses have a dual meaning: it is either unlimited Mtu size or Mtu reporting is not supported by the forwarder.

“MTU reporting is not supported by the forwarder” is not a concern. A forwarder that claims to implement FaceMgmt r90 must support MTU reporting.
The question is: is it possible that “MTU reporting is not supported by the face”?

Junxiao Shi wrote:

“MTU reporting is not supported by the forwarder” is not a concern. A forwarder that claims to implement FaceMgmt r90 must support MTU reporting.

How would a client know that?

The question is: is it possible that “MTU reporting is not supported by the face”?

Same thing. My concern is that there is no way to reliably distinguish between "unlimited MTU" and "MTU not reported for whatever reason".

nfdc comes together with NFD codebase and they are supposed to implement the same Management protocol.
Other clients can query NFD version and know management protocol version.

If there’s a possibility for a face to not support MTU reporting, it will make sense to distinguish between unlimited and undefined.
However, I doubt such a case exists.

Finally, “unlimited MTU” isn’t necessary. Effective MTU of every transport is at most MAX_NDN_PACKET_SIZE, since the transport won’t be able to receive packets longer than that. With that, MTU would be a required field.

In r91, I updated FaceMgmt with the results of our recent discussion in this issue. The Mtu field in a response contains the effective MTU of the face (with 0 indicating infinite). If this field is omitted, it indicates that the forwarder and/or face does not support MTU reporting.

Junxiao Shi wrote:

Other clients can query NFD version and know management protocol version.

I don't like this idea very much. It seems like it would require some strange mapping between NFD version and management protocol version (especially for development versions of NFD, where we use git commit hashes as a major part of the version number).

r92 extends the changes in r91 to FaceStatus.

In the NDN Platform Call on 2018-06-11, it was agreed to use Junxiao's proposal from note 43 to limit reported MTUs (even unlimited MTUs) to MAX_NDN_PACKET_SIZE.

In r93, FaceMgmt has been adjusted to use the effective MTU (capped at MAX_NDN_PACKET_SIZE) discussed in note 43.