Tuesday, August 13, 2013

Multiple NICs and You

Have you tried to configure more than one network card on more than one network in CentOS 6.x or RedHat 6.x yet? Well I did, and I was in for a surprise!
Turns out, that you cannot do it on CentOS easily. I tried this one several servers and even though the IP shows up, I could not ping it from the second network. It was a pain.
Working example is I have interface one configured as 10.20.0.100 netmask 255.255.255.0
I wanted interface two configured on 10.200.0.100 netmask 255.255.255.0
After going crazy verifying every setting in my /etc/sysconfig/network-scripts/ifgcfg-eth0 and ifcfg-eth1 files I found this blog post http://www.virtualizationteam.com/cloud/how-to-get-redhat-6-2-linux-second-nic-to-ping.html  that was an unrelated product, but the same root cause.
So after reading it i tried the settings they recommended and it worked, i did have to reboot to get it working fully, but that was all.

I then had another server we needed this on and I wanted to dig a little deeper, and make less edits, so i view the file /etc/sysctl.conf and this line stuck out to me

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

As an experiment I changed the 1 to a 0 for 
# Controls source route verification
net.ipv4.conf.default.rp_filter = 0
Reboot and POOF!! Both networks pingable from their respective subnets. 
So long story short, it wasnt a routing issue exactly but an issue with reverse packet filtering and how it routes packets out. 

2 comments:

  1. Can we get an explanation of what reverse packet filtering is and what function it is supposed to perform. Sure I could probably google that information, but I am lazy.

    ReplyDelete
  2. From what I was understanding it is when a packet goes out one interface, it is not allowed back on the second interface. This prevents packets from being forked/dropped/or labeled as bogus.
    The case here is the kernel is choosing the best outbound path for the packets and ignoring the second network from external sources, not answering there requests.

    ReplyDelete