Bug #3676: WebSocketTransport/StaticPropertiesNonLocalIpv4 RemoteUri mismatch - NFD - NDN project issue tracking system

Actions

Copy link

Bug #3676

closed

WebSocketTransport/StaticPropertiesNonLocalIpv4 RemoteUri mismatch

Added by Junxiao Shi over 8 years ago. Updated over 8 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Davide Pesavento

Category:

Faces

Target version:

v0.5

Start date:

07/18/2016

Due date:

% Done:

100%

Estimated time:

3.00 h

Description

Face/TestWebSocketTransport/StaticPropertiesNonLocalIpv4 test case picks an IP address from an available NIC, creates a WebSocket server to listen on an endpoint with this IP address, creates a WebSocket client to connect to the server endpoint, and then constructs a WebSocketTransport from a connect handle accepted by the WebSocket server.

The test case then expects the WebSocketTransport to report the initially picked IP address on both LocalUri and RemoteUri fields.

However, this fails in one instance https://travis-ci.org/yoursunny/ndn-cxx-breaks/jobs/145665660#L2147-L2150:

1468878712.221268 INFO: [WebSocketTransport] [id=0,local=ws://172.17.0.1:20070,remote=wsclient://10.128.0.98:36287] Creating transport
../tests/daemon/face/websocket-transport.t.cpp(243): error in "StaticPropertiesNonLocalIpv4": check transport->getRemoteUri().getHost() == address.to_string() failed [10.128.0.98 != 172.17.0.1]

The test case assumes the underlying socket of WebSocket client would choose the same IP address, given that IP address belongs to an available NIC.
However, this assumption isn't always true. Certain configuration of the IP routing table could cause a different IP address being chosen for the WebSocket client, so that RemoteUri seen on server-side connect handle would have an unexpected IP address.

Actions

Copy link

Updated by Junxiao Shi over 8 years ago

Only a subset of TravisCS builds have this problem.

Failed builds:

Successful builds:

I haven't found how to reproduce this IPv4 routing configuration in Vagrant.

Actions

Copy link

Updated by Junxiao Shi over 8 years ago

I made TravisCI reveal its IP settings with this .travis.yml:

sudo: required
language: generic
script:
  - ip link
  - ip addr
  - ip route

and triggered builds several times:

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP qlen 1000
    link/ether 42:01:0a:f0:00:a0 brd ff:ff:ff:ff:ff:ff
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP qlen 1000
    link/ether 42:01:0a:f0:00:a0 brd ff:ff:ff:ff:ff:ff
    inet 10.240.0.160/32 brd 10.240.0.160 scope global eth0
       valid_lft forever preferred_lft forever
$ ip route
default via 10.240.0.1 dev eth0 
default via 10.240.0.1 dev eth0  metric 100 
10.240.0.1 dev eth0  scope link

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP qlen 1000
    link/ether 42:01:0a:80:00:1a brd ff:ff:ff:ff:ff:ff
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP qlen 1000
    link/ether 42:01:0a:80:00:1a brd ff:ff:ff:ff:ff:ff
    inet 10.128.0.26/32 brd 10.128.0.26 scope global eth0
       valid_lft forever preferred_lft forever
$ ip route
default via 10.128.0.1 dev eth0 
default via 10.128.0.1 dev eth0  metric 100 
10.128.0.1 dev eth0  scope link

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP qlen 1000
    link/ether 42:01:0a:80:00:26 brd ff:ff:ff:ff:ff:ff
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP qlen 1000
    link/ether 42:01:0a:80:00:26 brd ff:ff:ff:ff:ff:ff
    inet 10.128.0.38/32 brd 10.128.0.38 scope global eth0
       valid_lft forever preferred_lft forever
$ ip route
default via 10.128.0.1 dev eth0 
default via 10.128.0.1 dev eth0  metric 100 
10.128.0.1 dev eth0  scope link

Although I can easily reproduce these settings in a Vagrant box, I still don't know where 172.17.0.1 comes from.

Actions

Copy link

Updated by Junxiao Shi over 8 years ago

note-2 .travis.yml is missing two lines to select trusty environment:

os: linux
dist: trusty

After adding them, some of the outputs are:

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 42:01:0a:f0:00:f9 brd ff:ff:ff:ff:ff:ff
3: lxcbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default 
    link/ether 3a:d3:65:da:ae:7c brd ff:ff:ff:ff:ff:ff
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default 
    link/ether 02:42:ac:ab:b8:69 brd ff:ff:ff:ff:ff:ff
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc mq state UP group default qlen 1000
    link/ether 42:01:0a:f0:00:f9 brd ff:ff:ff:ff:ff:ff
    inet 10.240.0.249/32 brd 10.240.0.249 scope global eth0
       valid_lft forever preferred_lft forever
3: lxcbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
    link/ether 3a:d3:65:da:ae:7c brd ff:ff:ff:ff:ff:ff
    inet 10.0.3.1/24 brd 10.0.3.255 scope global lxcbr0
       valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:ac:ab:b8:69 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
$ ifconfig
docker0   Link encap:Ethernet  HWaddr 02:42:ac:ab:b8:69  
          inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
eth0      Link encap:Ethernet  HWaddr 42:01:0a:f0:00:f9  
          inet addr:10.240.0.249  Bcast:10.240.0.249  Mask:255.255.255.255
          UP BROADCAST RUNNING MULTICAST  MTU:1460  Metric:1
          RX packets:2071 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2012 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:15774068 (15.7 MB)  TX bytes:271093 (271.0 KB)
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
lxcbr0    Link encap:Ethernet  HWaddr 3a:d3:65:da:ae:7c  
          inet addr:10.0.3.1  Bcast:10.0.3.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
$ ip route
default via 10.240.0.1 dev eth0 
10.0.3.0/24 dev lxcbr0  proto kernel  scope link  src 10.0.3.1 
10.240.0.1 dev eth0  scope link 
172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.0.1 
$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.240.0.1      0.0.0.0         UG    0      0        0 eth0
10.0.3.0        *               255.255.255.0   U     0      0        0 lxcbr0
10.240.0.1      *               255.255.255.255 UH    0      0        0 eth0
172.17.0.0      *               255.255.0.0     U     0      0        0 docker0
$ route -6
Kernel IPv6 routing table
Destination                    Next Hop                   Flag Met Ref Use If
::/0                           ::                         !n   -1  1     5 lo
::/0                           ::                         !n   -1  1     5 lo

I haven't figured out how to replicate this settings, but "docker" is a definite clue.

Actions

Copy link

Updated by Junxiao Shi over 8 years ago

It's worth noting that TcpTransport/StaticPropertiesNonLocalIpv4 test case has the same logic, and the RemoteUri field is correct.

Thus, this might as well be a bug in websocket++ 0.7.

Actions

Copy link

Updated by Davide Pesavento over 8 years ago

I can reproduce locally:

$ sudo modprobe dummy
$ sudo ifconfig dummy0 192.168.100.100
$ # still works
$ ./build/unit-tests-daemon -t Face/TestWebSocketTransport/StaticPropertiesNonLocalIpv4
Running 1 test case...
1474225798.881305 INFO: [WebSocketTransport] [id=0,local=ws://192.168.100.100:20070,remote=wsclient://192.168.100.100:49338] Creating transport

*** No errors detected

$ # add rule
$ sudo iptables -t nat -A POSTROUTING -s 192.168.100.0/24 ! -o dummy0 -j MASQUERADE
$ # now it fails
$ ./build/unit-tests-daemon -t Face/TestWebSocketTransport/StaticPropertiesNonLocalIpv4
Running 1 test case...
1474226205.818094 INFO: [WebSocketTransport] [id=0,local=ws://192.168.100.100:20070,remote=wsclient://10.0.2.7:49460] Creating transport
../tests/daemon/face/websocket-transport.t.cpp(249): error in "StaticPropertiesNonLocalIpv4": check transport->getRemoteUri().getHost() == address.to_string() failed [10.0.2.7 != 192.168.100.100]

*** 1 failure detected (1 failure expected) in test suite "Daemon Tests"

$ # flush chain
$ sudo iptables -t nat -F POSTROUTING
$ # works again
$ ./build/unit-tests-daemon -t Face/TestWebSocketTransport/StaticPropertiesNonLocalIpv4
Running 1 test case...
1474226294.476767 INFO: [WebSocketTransport] [id=0,local=ws://192.168.100.100:20070,remote=wsclient://192.168.100.100:49462] Creating transport

*** No errors detected

I have no idea why TcpTransport is not affected though.

$ sudo iptables -t nat -A POSTROUTING -s 192.168.100.0/24 ! -o dummy0 -j MASQUERADE
$ ./build/unit-tests-daemon -t Face/TestTcpTransport/StaticPropertiesNonLocalIpv4
Running 1 test case...
1474226629.201698 INFO: [TcpTransport] [id=0,local=tcp4://192.168.100.100:53778,remote=tcp4://192.168.100.100:7070] Creating transport

*** No errors detected

Actions

Copy link

Updated by Davide Pesavento over 8 years ago

FTR, the output of iptables-save on Travis:

# Generated by iptables-save v1.4.21 on Sun Sep 18 18:05:21 2016
*mangle
:PREROUTING ACCEPT [2601:15816017]
:INPUT ACCEPT [2601:15816017]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [2234:295890]
:POSTROUTING ACCEPT [2234:295890]
-A POSTROUTING -o lxcbr0 -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Sun Sep 18 18:05:22 2016
# Generated by iptables-save v1.4.21 on Sun Sep 18 18:05:22 2016
*nat
:PREROUTING ACCEPT [33:1980]
:INPUT ACCEPT [33:1980]
:OUTPUT ACCEPT [167:12678]
:POSTROUTING ACCEPT [167:12678]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 10.0.3.0/24 ! -d 10.0.3.0/24 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
COMMIT
# Completed on Sun Sep 18 18:05:22 2016
# Generated by iptables-save v1.4.21 on Sun Sep 18 18:05:22 2016
*filter
:INPUT ACCEPT [2558:15807616]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [2200:294345]
:DOCKER - [0:0]
:DOCKER-ISOLATION - [0:0]
:sshguard - [0:0]
-A INPUT -j sshguard
-A INPUT -i lxcbr0 -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i lxcbr0 -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i lxcbr0 -p tcp -m tcp --dport 67 -j ACCEPT
-A INPUT -i lxcbr0 -p udp -m udp --dport 67 -j ACCEPT
-A FORWARD -j DOCKER-ISOLATION
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o lxcbr0 -j ACCEPT
-A FORWARD -i lxcbr0 -j ACCEPT
-A DOCKER-ISOLATION -j RETURN
COMMIT
# Completed on Sun Sep 18 18:05:22 2016

Actions

Copy link

Updated by Davide Pesavento over 8 years ago

I have no idea why TcpTransport is not affected though.

I figured it out.

TcpTransport tests are done from the client side (the transport is created from the socket that performs the connect), so the transport sees the "right" addresses (i.e. non-NATed) and puts 192.168.100.100 in both local and remote URIs. On the other hand, our WebSocketTransport models the server side only, so its tests are done from the server side, which only sees the NATed IP of the client, and that's what it uses to configure the remote URI of the transport.

As you can see below, in both cases the server sees the NATed IP address of the client (10.0.2.7), so the behavior is actually consistent.

# TcpTransport
$ ss -n4t
State       Recv-Q Send-Q                 Local Address:Port                              Peer Address:Port
ESTAB       0      0                    192.168.100.100:7070                                  10.0.2.7:53830
ESTAB       0      0                    192.168.100.100:53830                          192.168.100.100:7070

# WebSocketTransport
$ ss -n4t
State       Recv-Q Send-Q                 Local Address:Port                              Peer Address:Port
ESTAB       0      0                    192.168.100.100:49520                          192.168.100.100:20070
ESTAB       0      0                    192.168.100.100:20070                                 10.0.2.7:49520

Actions

Copy link

Updated by Junxiao Shi over 8 years ago

Assignee set to Weiwei Liu
Target version set to v0.5

According to note-5 analysis, it's wrong to assume RemoteUri has the same IP address.
Thus, this test case can just check RemoteUri has the proper scheme and port number, and should not check IP address portion.

This issue is assigned to Weiwei who authored this test case in nfd:commit:93606238531f33ce8a4daf1279741731e595f57f.

Actions

Copy link