Wifi + Ethernet Roulette

snickerdoodle general
1

Our custom baseboard has an Ethernet PHY, and of course the snickerdoodle has WiFi on-board. When both are enabled (under Linux) it's a crap-shoot as to which one (never both) will end up listening for TCP connections. This becomes a real issue when both I/Fs have dynamic IP addresses and the console is unavailable.
I'm sure there is a way to convince Linux to listen on both interfaces, but it's not obvious to me, and there's too much noise on Google to get any real help there.
This will likely be an issue for the Pi-Smasher. Anyone have this one figured out?
Thanks,

-Nick
2
Hi Nick,

We will refer this one to Bush, our resident Linux expert as he has managed to get 4 network interfaces working on the piSmashers.

AFAIK, the bind() and listen() calls in any given server thread are associated with a specific IP address so listening on multiple interfaces is something that has to be supported either by the TCP server itself or whatever dispatches the connections in the "internet super server" type case.

Hopefully, Bush will be able to shine some light on what configurations are needed to support this. You did not say what server it was that you were trying to reach over multiple ports?

-Jamil




On Tuesday, August 29, 2017 at 4:32:31 PM UTC-7, Nick Burkitt wrote:

Our custom baseboard has an Ethernet PHY, and of course the snickerdoodle has WiFi on-board. When both are enabled (under Linux) it's a crap-shoot as to which one (never both) will end up listening for TCP connections. This becomes a real issue when both I/Fs have dynamic IP addresses and the console is unavailable.
I'm sure there is a way to convince Linux to listen on both interfaces, but it's not obvious to me, and there's too much noise on Google to get any real help there.
This will likely be an issue for the Pi-Smasher. Anyone have this one figured out?
Thanks,

-Nick
3
It seems that inetd definitely has this capability: http://manpages.ubuntu.com/manpages/zesty/man8/inetd.8.html

"For internet services, the first field of the line may also have a host 
address specifier prefixed to it, separated from the service name by a
colon.  If this is done, the string before the colon in the first field
indicates what local address inetd should use when listening for that
service.  Multiple local addresses can be specified on the same line,
separated by commas.  Numeric IP addresses in dotted-quad notation can be
used as well as symbolic hostnames.  Symbolic hostnames are looked up
using getaddrinfo().  If a hostname has multiple address mappings, inetd
creates a socket to listen on each address."

and so does sshd: http://manpages.ubuntu.com/manpages/zesty/en/man5/sshd_config.5.html

"ListenAddress 
             Specifies the local addresses sshd(8) should listen on.  The
             following forms may be used:

               <b>ListenAddress</b> <u>host</u>|<u>IPv4_addr</u>|<u>IPv6_addr</u>
               <b>ListenAddress</b> <u>host</u>|<u>IPv4_addr</u>:<u>port</u>
               <b>ListenAddress</b> [<u>host</u>|<u>IPv6_addr</u>]:<u>port</u>

         If <u>port</u> is not specified, sshd will listen on the address and all
         <b>Port</b> options specified.  The default is to listen on all local
         addresses.  Multiple <b>ListenAddress</b> options are permitted.&quot;</span></pre><br><br>But I&#039;m no Linux expert.<br><br>-Jamil<br><br><br><br>On Tuesday, August 29, 2017 at 5:26:33 PM UTC-7, weath...@krtkl.com wrote:<blockquote class="gmail_quote" style="margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex"><div dir="ltr">Hi Nick,<br><br>We will refer this one to Bush, our resident Linux expert as he has managed to get 4 network interfaces working on the piSmashers.<br><br>AFAIK, the bind() and listen() calls in any given server thread are associated with a specific IP address so listening on multiple interfaces is something that has to be supported either by the TCP server itself or whatever dispatches the connections in the &quot;internet super server&quot; type case.<br><br>Hopefully, Bush will be able to shine some light on what configurations are needed to support this.   You did not say what server it was that you were trying to reach over multiple ports?<br><br>-Jamil<br><br><br><br><br>On Tuesday, August 29, 2017 at 4:32:31 PM UTC-7, Nick Burkitt wrote:<blockquote class="gmail_quote" style="margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br>Our custom baseboard has an Ethernet PHY, and of course the snickerdoodle has WiFi on-board. When both are enabled (under Linux) it&#039;s a crap-shoot as to which one (never both) will end up listening for TCP connections. This becomes a real issue when both I/Fs have dynamic IP addresses and the console is unavailable.<div>I&#039;m sure there is a way to convince Linux to listen on both interfaces, but it&#039;s not obvious to me, and there&#039;s too much noise on Google to get any real help there.</div><div>This will likely be an issue for the Pi-Smasher. Anyone have this one figured out?</div><div>Thanks,</div><div><br></div><div>-Nick</div></div></blockquote></div></blockquote></div>
5
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
6
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
7
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
8
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
9
I think I've been able to duplicate the behavior you're experiencing. When both interfaces are up I can only ping one interface at a time, or rather, only one interface will give a proper ping reply. What it looks like is happening (from tcpdump) is that the ping requests are being received and then routed (seemingly arbitrarily) indiscriminately to either interface. This causes the replys to be sourced from an IP address that the pinging host does not expect, causing the failure. I believe the SSH service is being handles similarly. This aligns with the behavior you're seeing with the interfaces properly connecting when they are on different networks/subnets. Ultimately, it looks like a routing issue.

The network interface that points to the gateway in the routing table is the one that is externally accessible:

snickerdoodle@snickerdoodle:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.111.1 0.0.0.0 UG 0 0 0 eth0
10.0.110.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0

This means that all traffic will be routed through eth0 by default. We need to set it up so that the traffic is routed through the interface from which it was received. I believe I'm using a similar configuration as you (both wired and wireless on same subnet via DHCP) I'm still trying to find a workable configuration with the setup that I have but I wanted to get back to you with what I've found so far. Let me know if this seems like I'm on the right path or if this looks nothing like the behavior you're seeing.

On Tuesday, September 5, 2017 at 9:34:31 AM UTC-7, Nick Burkitt wrote:
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
10
Yes, that description matches the symptoms I'm seeing. And it (kind of) makes sense that replies would be sent using the default route. Thanks for the update. I'm keeping my fingers crossed... :-)

On Tuesday, September 5, 2017 at 2:57:10 PM UTC-7, Bush wrote:
I think I've been able to duplicate the behavior you're experiencing. When both interfaces are up I can only ping one interface at a time, or rather, only one interface will give a proper ping reply. What it looks like is happening (from tcpdump) is that the ping requests are being received and then routed (seemingly arbitrarily) indiscriminately to either interface. This causes the replys to be sourced from an IP address that the pinging host does not expect, causing the failure. I believe the SSH service is being handles similarly. This aligns with the behavior you're seeing with the interfaces properly connecting when they are on different networks/subnets. Ultimately, it looks like a routing issue.

The network interface that points to the gateway in the routing table is the one that is externally accessible:

snickerdoodle@snickerdoodle:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.111.1 0.0.0.0 UG 0 0 0 eth0
10.0.110.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0

This means that all traffic will be routed through eth0 by default. We need to set it up so that the traffic is routed through the interface from which it was received. I believe I'm using a similar configuration as you (both wired and wireless on same subnet via DHCP) I'm still trying to find a workable configuration with the setup that I have but I wanted to get back to you with what I've found so far. Let me know if this seems like I'm on the right path or if this looks nothing like the behavior you're seeing.

On Tuesday, September 5, 2017 at 9:34:31 AM UTC-7, Nick Burkitt wrote:
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
11
And, it suggests some useful Google searches. Like the one that produced this result: https://unix.stackexchange.com/questions/4420/reply-on-same-interface-as-incoming

On Tuesday, September 5, 2017 at 3:15:34 PM UTC-7, Nick Burkitt wrote:
Yes, that description matches the symptoms I'm seeing. And it (kind of) makes sense that replies would be sent using the default route. Thanks for the update. I'm keeping my fingers crossed... :-)

On Tuesday, September 5, 2017 at 2:57:10 PM UTC-7, Bush wrote:
I think I've been able to duplicate the behavior you're experiencing. When both interfaces are up I can only ping one interface at a time, or rather, only one interface will give a proper ping reply. What it looks like is happening (from tcpdump) is that the ping requests are being received and then routed (seemingly arbitrarily) indiscriminately to either interface. This causes the replys to be sourced from an IP address that the pinging host does not expect, causing the failure. I believe the SSH service is being handles similarly. This aligns with the behavior you're seeing with the interfaces properly connecting when they are on different networks/subnets. Ultimately, it looks like a routing issue.

The network interface that points to the gateway in the routing table is the one that is externally accessible:

snickerdoodle@snickerdoodle:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.111.1 0.0.0.0 UG 0 0 0 eth0
10.0.110.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0

This means that all traffic will be routed through eth0 by default. We need to set it up so that the traffic is routed through the interface from which it was received. I believe I'm using a similar configuration as you (both wired and wireless on same subnet via DHCP) I'm still trying to find a workable configuration with the setup that I have but I wanted to get back to you with what I've found so far. Let me know if this seems like I'm on the right path or if this looks nothing like the behavior you're seeing.

On Tuesday, September 5, 2017 at 9:34:31 AM UTC-7, Nick Burkitt wrote:
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
12
I'm still nowhere on this. The suggestions I've been able to find have not had any effect, and I don't have the time right now to become a Linux networking expert. If you come up with anything, please let me know.
Thanks,

-Nick

On Tuesday, September 5, 2017 at 3:18:12 PM UTC-7, Nick Burkitt wrote:
And, it suggests some useful Google searches. Like the one that produced this result: https://unix.stackexchange.com/questions/4420/reply-on-same-interface-as-incoming

On Tuesday, September 5, 2017 at 3:15:34 PM UTC-7, Nick Burkitt wrote:
Yes, that description matches the symptoms I'm seeing. And it (kind of) makes sense that replies would be sent using the default route. Thanks for the update. I'm keeping my fingers crossed... :-)

On Tuesday, September 5, 2017 at 2:57:10 PM UTC-7, Bush wrote:
I think I've been able to duplicate the behavior you're experiencing. When both interfaces are up I can only ping one interface at a time, or rather, only one interface will give a proper ping reply. What it looks like is happening (from tcpdump) is that the ping requests are being received and then routed (seemingly arbitrarily) indiscriminately to either interface. This causes the replys to be sourced from an IP address that the pinging host does not expect, causing the failure. I believe the SSH service is being handles similarly. This aligns with the behavior you're seeing with the interfaces properly connecting when they are on different networks/subnets. Ultimately, it looks like a routing issue.

The network interface that points to the gateway in the routing table is the one that is externally accessible:

snickerdoodle@snickerdoodle:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.111.1 0.0.0.0 UG 0 0 0 eth0
10.0.110.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0

This means that all traffic will be routed through eth0 by default. We need to set it up so that the traffic is routed through the interface from which it was received. I believe I'm using a similar configuration as you (both wired and wireless on same subnet via DHCP) I'm still trying to find a workable configuration with the setup that I have but I wanted to get back to you with what I've found so far. Let me know if this seems like I'm on the right path or if this looks nothing like the behavior you're seeing.

On Tuesday, September 5, 2017 at 9:34:31 AM UTC-7, Nick Burkitt wrote:
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
13
You're looking for VRFs or (more extremely) network namespaces. Either will allow you to bring an interface into a separate routing table which will have separate default gateways.

https://www.kernel.org/doc/Documentation/networking/vrf.txt


On Tuesday, September 12, 2017 at 1:14:07 PM UTC-4, Nick Burkitt wrote:
I'm still nowhere on this. The suggestions I've been able to find have not had any effect, and I don't have the time right now to become a Linux networking expert. If you come up with anything, please let me know.
Thanks,

-Nick

On Tuesday, September 5, 2017 at 3:18:12 PM UTC-7, Nick Burkitt wrote:
And, it suggests some useful Google searches. Like the one that produced this result: https://unix.stackexchange.com/questions/4420/reply-on-same-interface-as-incoming

On Tuesday, September 5, 2017 at 3:15:34 PM UTC-7, Nick Burkitt wrote:
Yes, that description matches the symptoms I'm seeing. And it (kind of) makes sense that replies would be sent using the default route. Thanks for the update. I'm keeping my fingers crossed... :-)

On Tuesday, September 5, 2017 at 2:57:10 PM UTC-7, Bush wrote:
I think I've been able to duplicate the behavior you're experiencing. When both interfaces are up I can only ping one interface at a time, or rather, only one interface will give a proper ping reply. What it looks like is happening (from tcpdump) is that the ping requests are being received and then routed (seemingly arbitrarily) indiscriminately to either interface. This causes the replys to be sourced from an IP address that the pinging host does not expect, causing the failure. I believe the SSH service is being handles similarly. This aligns with the behavior you're seeing with the interfaces properly connecting when they are on different networks/subnets. Ultimately, it looks like a routing issue.

The network interface that points to the gateway in the routing table is the one that is externally accessible:

snickerdoodle@snickerdoodle:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.111.1 0.0.0.0 UG 0 0 0 eth0
10.0.110.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0

This means that all traffic will be routed through eth0 by default. We need to set it up so that the traffic is routed through the interface from which it was received. I believe I'm using a similar configuration as you (both wired and wireless on same subnet via DHCP) I'm still trying to find a workable configuration with the setup that I have but I wanted to get back to you with what I've found so far. Let me know if this seems like I'm on the right path or if this looks nothing like the behavior you're seeing.

On Tuesday, September 5, 2017 at 9:34:31 AM UTC-7, Nick Burkitt wrote:
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
14
Hi Daniel.

Thanks - are VRFs supported in kernel version 4.4? From what I can see, iproute2 added support for at least the vrf keyword in 4.7. We're using Ubuntu 14.04 still - we're not in a position to upgrade to 16.04 at the moment.

-Nick

On Tuesday, September 12, 2017 at 4:08:04 PM UTC-7, Daniel Smith wrote:
You're looking for VRFs or (more extremely) network namespaces. Either will allow you to bring an interface into a separate routing table which will have separate default gateways.



On Tuesday, September 12, 2017 at 1:14:07 PM UTC-4, Nick Burkitt wrote:
I'm still nowhere on this. The suggestions I've been able to find have not had any effect, and I don't have the time right now to become a Linux networking expert. If you come up with anything, please let me know.
Thanks,

-Nick

On Tuesday, September 5, 2017 at 3:18:12 PM UTC-7, Nick Burkitt wrote:
And, it suggests some useful Google searches. Like the one that produced this result: https://unix.stackexchange.com/questions/4420/reply-on-same-interface-as-incoming

On Tuesday, September 5, 2017 at 3:15:34 PM UTC-7, Nick Burkitt wrote:
Yes, that description matches the symptoms I'm seeing. And it (kind of) makes sense that replies would be sent using the default route. Thanks for the update. I'm keeping my fingers crossed... :-)

On Tuesday, September 5, 2017 at 2:57:10 PM UTC-7, Bush wrote:
I think I've been able to duplicate the behavior you're experiencing. When both interfaces are up I can only ping one interface at a time, or rather, only one interface will give a proper ping reply. What it looks like is happening (from tcpdump) is that the ping requests are being received and then routed (seemingly arbitrarily) indiscriminately to either interface. This causes the replys to be sourced from an IP address that the pinging host does not expect, causing the failure. I believe the SSH service is being handles similarly. This aligns with the behavior you're seeing with the interfaces properly connecting when they are on different networks/subnets. Ultimately, it looks like a routing issue.

The network interface that points to the gateway in the routing table is the one that is externally accessible:

snickerdoodle@snickerdoodle:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.111.1 0.0.0.0 UG 0 0 0 eth0
10.0.110.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0

This means that all traffic will be routed through eth0 by default. We need to set it up so that the traffic is routed through the interface from which it was received. I believe I'm using a similar configuration as you (both wired and wireless on same subnet via DHCP) I'm still trying to find a workable configuration with the setup that I have but I wanted to get back to you with what I've found so far. Let me know if this seems like I'm on the right path or if this looks nothing like the behavior you're seeing.

On Tuesday, September 5, 2017 at 9:34:31 AM UTC-7, Nick Burkitt wrote:
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
15
I don't think VRFs are present in that version of Ubuntu but network namespaces should provide similar virtualized route tables (amongst other things you don't care about) in that version or older.

On Wednesday, September 13, 2017 at 6:05:58 PM UTC-4, Nick Burkitt wrote:
Hi Daniel.

Thanks - are VRFs supported in kernel version 4.4? From what I can see, iproute2 added support for at least the vrf keyword in 4.7. We're using Ubuntu 14.04 still - we're not in a position to upgrade to 16.04 at the moment.

-Nick

On Tuesday, September 12, 2017 at 4:08:04 PM UTC-7, Daniel Smith wrote:
You're looking for VRFs or (more extremely) network namespaces. Either will allow you to bring an interface into a separate routing table which will have separate default gateways.



On Tuesday, September 12, 2017 at 1:14:07 PM UTC-4, Nick Burkitt wrote:
I'm still nowhere on this. The suggestions I've been able to find have not had any effect, and I don't have the time right now to become a Linux networking expert. If you come up with anything, please let me know.
Thanks,

-Nick

On Tuesday, September 5, 2017 at 3:18:12 PM UTC-7, Nick Burkitt wrote:
And, it suggests some useful Google searches. Like the one that produced this result: https://unix.stackexchange.com/questions/4420/reply-on-same-interface-as-incoming

On Tuesday, September 5, 2017 at 3:15:34 PM UTC-7, Nick Burkitt wrote:
Yes, that description matches the symptoms I'm seeing. And it (kind of) makes sense that replies would be sent using the default route. Thanks for the update. I'm keeping my fingers crossed... :-)

On Tuesday, September 5, 2017 at 2:57:10 PM UTC-7, Bush wrote:
I think I've been able to duplicate the behavior you're experiencing. When both interfaces are up I can only ping one interface at a time, or rather, only one interface will give a proper ping reply. What it looks like is happening (from tcpdump) is that the ping requests are being received and then routed (seemingly arbitrarily) indiscriminately to either interface. This causes the replys to be sourced from an IP address that the pinging host does not expect, causing the failure. I believe the SSH service is being handles similarly. This aligns with the behavior you're seeing with the interfaces properly connecting when they are on different networks/subnets. Ultimately, it looks like a routing issue.

The network interface that points to the gateway in the routing table is the one that is externally accessible:

snickerdoodle@snickerdoodle:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.111.1 0.0.0.0 UG 0 0 0 eth0
10.0.110.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0

This means that all traffic will be routed through eth0 by default. We need to set it up so that the traffic is routed through the interface from which it was received. I believe I'm using a similar configuration as you (both wired and wireless on same subnet via DHCP) I'm still trying to find a workable configuration with the setup that I have but I wanted to get back to you with what I've found so far. Let me know if this seems like I'm on the right path or if this looks nothing like the behavior you're seeing.

On Tuesday, September 5, 2017 at 9:34:31 AM UTC-7, Nick Burkitt wrote:
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?
16
Thanks, I'll look into it when I have some time...

-Nick

On Wednesday, September 20, 2017 at 6:38:29 PM UTC-7, Daniel Smith wrote:
I don't think VRFs are present in that version of Ubuntu but network namespaces should provide similar virtualized route tables (amongst other things you don't care about) in that version or older.

On Wednesday, September 13, 2017 at 6:05:58 PM UTC-4, Nick Burkitt wrote:
Hi Daniel.

Thanks - are VRFs supported in kernel version 4.4? From what I can see, iproute2 added support for at least the vrf keyword in 4.7. We're using Ubuntu 14.04 still - we're not in a position to upgrade to 16.04 at the moment.

-Nick

On Tuesday, September 12, 2017 at 4:08:04 PM UTC-7, Daniel Smith wrote:
You're looking for VRFs or (more extremely) network namespaces. Either will allow you to bring an interface into a separate routing table which will have separate default gateways.



On Tuesday, September 12, 2017 at 1:14:07 PM UTC-4, Nick Burkitt wrote:
I'm still nowhere on this. The suggestions I've been able to find have not had any effect, and I don't have the time right now to become a Linux networking expert. If you come up with anything, please let me know.
Thanks,

-Nick

On Tuesday, September 5, 2017 at 3:18:12 PM UTC-7, Nick Burkitt wrote:
And, it suggests some useful Google searches. Like the one that produced this result: https://unix.stackexchange.com/questions/4420/reply-on-same-interface-as-incoming

On Tuesday, September 5, 2017 at 3:15:34 PM UTC-7, Nick Burkitt wrote:
Yes, that description matches the symptoms I'm seeing. And it (kind of) makes sense that replies would be sent using the default route. Thanks for the update. I'm keeping my fingers crossed... :-)

On Tuesday, September 5, 2017 at 2:57:10 PM UTC-7, Bush wrote:
I think I've been able to duplicate the behavior you're experiencing. When both interfaces are up I can only ping one interface at a time, or rather, only one interface will give a proper ping reply. What it looks like is happening (from tcpdump) is that the ping requests are being received and then routed (seemingly arbitrarily) indiscriminately to either interface. This causes the replys to be sourced from an IP address that the pinging host does not expect, causing the failure. I believe the SSH service is being handles similarly. This aligns with the behavior you're seeing with the interfaces properly connecting when they are on different networks/subnets. Ultimately, it looks like a routing issue.

The network interface that points to the gateway in the routing table is the one that is externally accessible:

snickerdoodle@snickerdoodle:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.0.111.1 0.0.0.0 UG 0 0 0 eth0
10.0.110.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan1
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.0.111.0 0.0.0.0 255.255.255.0 U 0 0 0 wlan0

This means that all traffic will be routed through eth0 by default. We need to set it up so that the traffic is routed through the interface from which it was received. I believe I'm using a similar configuration as you (both wired and wireless on same subnet via DHCP) I'm still trying to find a workable configuration with the setup that I have but I wanted to get back to you with what I've found so far. Let me know if this seems like I'm on the right path or if this looks nothing like the behavior you're seeing.

On Tuesday, September 5, 2017 at 9:34:31 AM UTC-7, Nick Burkitt wrote:
An additional data point - if the two interfaces are connected to different networks, both will respond to connection attempts. We just need that to happen when they're both connected to the same network.

On Friday, September 1, 2017 at 1:20:44 PM UTC-7, Nick Burkitt wrote:
Any additional thoughts on this question?

On Wednesday, August 30, 2017 at 9:44:15 AM UTC-7, Nick Burkitt wrote:
No, it's the other way around. Both interfaces are able to obtain DHCP addresses, but only one of the two will respond to ssh, ping, ftp, or anything else.

On Tuesday, August 29, 2017 at 9:12:58 PM UTC-7, Bush wrote:
Just to clarify, you are able to SSH in to either interface but only 1 interface is successfully getting a DHCP provided IP address? If this is the case it's likely an issue with the /etc/dhcp/dhclient.conf. Is only one interface visible (pingable) from the outside at a time?