Setting up a Software Bridge on Top of a Teamed Device on RHEL 7

Posted on by

We are going to setup a bridged network interface on top of a teamed link.

Before We Begin

We use a RHEL 7.1 virtual machine with a teamed interface that we configured some time ago.

# nmcli c
NAME     UUID                                  TYPE            DEVICE
slave2   eabb15cc-2bb7-4fc1-a52f-47268137cb09  802-3-ethernet  enp0s8
slave1   6d420cc6-cba4-4561-b9ae-17af025987ce  802-3-ethernet  enp0s17
myteam0  9691a92f-03dc-48b2-951d-a2b73721f8bc  team            myteam0
# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s8: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master myteam0 state UP qlen 1000
    link/ether 08:00:27:ff:72:02 brd ff:ff:ff:ff:ff:ff
3: enp0s17: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master myteam0 state UP qlen 1000
    link/ether 08:00:27:ff:72:02 brd ff:ff:ff:ff:ff:ff
4: myteam0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 08:00:27:ff:72:02 brd ff:ff:ff:ff:ff:ff
    inet 10.8.8.72/24 brd 10.8.8.255 scope global myteam0
       valid_lft forever preferred_lft forever
    inet6 fc00::a:b:c:72/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:feff:7202/64 scope link
       valid_lft forever preferred_lft forever

Configure Software Bridge

Be advised that we use console and not Secure Shell (SSH). If we misconfigure something, network connection might and likely will be dropped.

Install the bridge-utils package:

# yum install -y bridge-utils

Add a new mybr0 connection:

# nmcli c add type bridge con-name mybr0 ifname mybr0

Modify the newly created mybr0 connection to match our configuration (see the teamed configuration for IPs etc):

# nmcli c mod mybr0 ipv4.method manual ipv6.method manual \
  ipv4.addresses 10.8.8.72/24 ipv4.gateway 10.8.8.70 \
  ipv4.dns 10.8.8.70 ipv6.addresses fc00::a:b:c:72/64 \
  bridge.stp no

Verify:

# nmcli c
NAME     UUID                                  TYPE            DEVICE
slave2   eabb15cc-2bb7-4fc1-a52f-47268137cb09  802-3-ethernet  enp0s8
slave1   6d420cc6-cba4-4561-b9ae-17af025987ce  802-3-ethernet  enp0s17
myteam0  9691a92f-03dc-48b2-951d-a2b73721f8bc  team            myteam0
mybr0    558eb416-de58-4b44-a545-8959cdc81e66  bridge          mybr0

Change to:

# cd /etc/sysconfig/network-scripts/

Open ifcfg-myteam0 for editing. Add the following line at the end of the file:

BRIDGE=mybr0

Also remove IP address configuration as it’s now set up on the bridge. We can use sed:

# sed -i '/IPADDR/d;/PREFIX/d;/GATEWAY/d;/DNS/d;/IPV6ADDR/d' ./ifcfg-myteam0

Disable DAD on the bridged interface:

# echo "net.ipv6.conf.mybr0.accept_dad=0" >>/etc/sysctl.d/dad.conf
# sysctl -w net.ipv6.conf.mybr0.accept_dad=0

Network Manager does not support bridges on bonded or teamed interfaces:

# systemctl disable NetworkManager && systemctl stop NetworkManager

The network service should be started and enabled:

# systemctl enable network && systemctl restart network

Verify:

# brctl show
bridge name     bridge id               STP enabled     interfaces
mybr0           8000.080027ff7201       no              myteam0
# ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s8: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master myteam0 state UP qlen 1000
    link/ether 08:00:27:ff:72:01 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a00:27ff:feff:7201/64 scope link
       valid_lft forever preferred_lft forever
3: enp0s17: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master myteam0 state UP qlen 1000
    link/ether 08:00:27:ff:72:01 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a00:27ff:feff:7201/64 scope link
       valid_lft forever preferred_lft forever
6: myteam0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master mybr0 state UP
    link/ether 08:00:27:ff:72:01 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::a00:27ff:feff:7201/64 scope link
       valid_lft forever preferred_lft forever
7: mybr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 08:00:27:ff:72:01 brd ff:ff:ff:ff:ff:ff
    inet 10.8.8.72/24 brd 10.8.8.255 scope global mybr0
       valid_lft forever preferred_lft forever
    inet6 fc00::a:b:c:72/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:feff:7201/64 scope link
       valid_lft forever preferred_lft forever

16 thoughts on “Setting up a Software Bridge on Top of a Teamed Device on RHEL 7”

  1. Hi
    I had to do “nmcli dev disconnect myteam0” before stopping NetworkManager and starting network. It failed with eno devices disapearing and bridge in wired state.

    Reply

  2. Hello Tomas, After both Teaming and now placing Bridge on the Team, can I now install Centos 7.3 KVM after doing these tasks? what will happen to the KVM Bridge that gets automatically added during KVM installation. Or should I install KVM First ? destroy the default bridge and start from there, been searching all over for a complete solution on this with no luck

    Thanks
    Ray

    Reply

    • Hi Ray, I’m not sure I understand your question. Are you setting up teaming and bridging on a KVM hypervisor, and are asking about installing a KVM guest on said hypervisor?

      What I usually do is I set up a KVM hypervisor first, it comes with a NAT interface which I don’t really need, so I get that disabled and configure a network bridge. Once that’s done I start creating KVM guests.

  3. Thanks for quick response and great site! I want to set up a KVM Virtual Host server to provide VMS as normal I just don’t know what I should do first Like
    Install KVM first
    or
    Create Network Team first ( 2 1GB Ethernet interfaces )
    Then create software Bridge next as you described
    Then install KVM?
    Thank you Hope that makes more sense
    Ray

    Reply

    • Tom Had a Problem with congiuration can I follow These Steps
      1 Create Network Team ( 2 1GB Nics)
      2 Install KVM Hypervisor
      3 Remove Default KVM Bridge Then Create my own Network Bridge
      4 Create VMs

      Thanks Tom
      Ray

  4. Thank you I will do that, and thanks again to The Best Linux Web Site I have Found in Years! Information you provide here helps others, on Difficult Topics, That I can speak for myself is Not covered by Major CBT Training Courses and a ton of Linux Books I have bought last several years!!!

    Thanks For Caring
    Ray

    Reply

  5. steps you provided for bridge over team didnt worked for me. I had to also remove some parameters from team slaves, there was some dns configuration which does not worked. I configure teaming with your steps.

    Reply

    • The steps do work on the RHEL version that’s mentioned in the article. It may however not work if you have a different version of the OS. Out of curiosity, which parameters did you have to remove from the slaves?

    • I am using centos 7.0, after initial configuration of teamming, network manager generate team slaves as:
      BOOTPROTO=dhcp
      DEFROUTE=yes
      PEERDNS=yes
      PEERROUTES=yes
      IPV4_FAILURE_FATAL=no
      IPV6INIT=yes
      IPV6_AUTOCONF=yes
      IPV6_DEFROUTE=yes
      IPV6_PEERDNS=yes
      IPV6_PEERROUTES=yes
      IPV6_FAILURE_FATAL=no
      #____ I always delete everything above this line ____
      NAME=t1
      UUID=da8dc162-7aa5-4748-b95e-2bf23a5f3a9a
      DEVICE=eno16777736
      ONBOOT=yes
      TEAM_MASTER=team0
      DEVICETYPE=TeamPort

      I think the biggest problem is BOOTPROTO=dhcp as i see dhcp timeouts. in syslog. I always having problem to configure it. It is always 5 minutes of configuration and 15 minutes troubleshooting, so I am keeping always ifcgs as slim as possible.

  6. yes, you are right! BOOTPROTO=none helped, I have tested today. If there is BOOTPROTO=dhcp there are lot of timeouts, but after 5 minutes of waiting it is also working, but i have never patience to wait. On exam 5 minutes waiting (on every restart of network) is probably difference between fail or win. Do you noticed same timeouts with your configuration?

    Feb 3 14:51:01 srv1 dhclient[4587]: DHCPDISCOVER on eno33554960 to 255.255.255.255 port 67 interval 11 (xid=0x32fa4740)
    Feb 3 14:51:12 srv1 dhclient[4587]: No DHCPOFFERS received.
    Feb 3 14:51:12 srv1 network: Determining IP information for eno33554960… failed.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *