Bridged Networks for OpenVZ in ALTLinux
Let's asume we have a computer with 2 network cards, one connected to internet router, has IP address 5.88.52.189/27 and it's default route should gateway via 5.88.52.161. And secord network connected to local area network 10.61.0.0/24. We want to create multiple OpenVZ Virtual Environments there with 2 virtual cards each, one with public IP from 5.88.52.0/27 range and other connected to local network.
OpenVZ has different network implementations for Virtual Environments. Default is venet, which is fastest, easiest and most secure.
Another option is veth, and that grants to VE right to have it's own routing table, and assign ip addresses to these interfaces.
And third option is granting access to hardware NIC (or virtual device, like VLAN, tun or tap) to VE, in that case it will be exclusively visible and manageable by this VE.
Creating bridges in HE
To use bridged networks in ALTLinux you should do the following:
- In ALTLinux interfaces with names eth0, eth1... are deprecated. There is no way to ensure their names and order are preserved on reboots and on kernel upgrades. So they should be renames using /etc/iftab to logical names, like wan, lan... To use bridges we must ensure that this table defines interfaces not by macaddresses, but by businfo or combination of businfo and other characteristics, which can be obtained by running ethtool -i:
# ethtool -i eth0
driver: forcedeth
version: 0.60
firmware-version:
bus-info: 0000:00:08.0
# ethtool -i msgs
driver: forcedeth
version: 0.60
firmware-version:
bus-info: 0000:00:09.0
# cat <<EOF > /etc/iftab
wan driver forcedeth businfo 0000:00:08.0 mac 00:30:48:7a:70:fe
lan driver forcedeth businfo 0000:00:09.0 mac 00:30:48:7a:70:ff
EOF
- Network in ALTLiunx is managed by etcnet package. Let's look at it's configuration:
# cd /etc/net/ifaces
# ls
default lo lan wan unknown venet0
directories lan and wan are created to store configurations for these 2 NICs. They can be just copies of directories eth0 and eth1 created for these NICs by installer, but with line
TYPE=eth
added to their {lan,wan}/options files.
- Now we create 2 directiries for bridges brwan and brlan:
# mkdir brwan brlan
# echo 'stp AUTO off' > brwan/brctl
# echo '5.88.52.189/27 broadcast 5.88.52.255 dev brwan' > brwan/ipv4address
# echo 'default via 5.88.52.161' > brwan/ipv4route
# cat <<EOF >brwan/options
BOOTPROTO=static
TYPE=bri
HOST='wan'
ONBOOT=yes
EOF
# echo 'stp AUTO off' > brlan/brctl
# echo '10.61.0.15 broadcast 10.61.0.255 dev brlan' > brlan/ipv4address
# echo '10.8.0.0/24 dev brlan' > brlan/ipv4route
# cat <<EOF >brlan/options
BOOTPROTO=static
TYPE=bri
HOST='wan'
ONBOOT=yes
EOF
- Now we should remove ip addresses and routes from lan/wan interfaces:
# echo '0.0.0.0/0' > lan/ipv4address
# echo '0.0.0.0/0' > wan/ipv4address
# rm -f wan/ipv4route
# rm -f lan/ipv4route
- Now it should be safe to /sbin/service network restart, but if you connected by ssh, better restart first networking for interface other then you connected by. Assume I'm connected by eth1:
# ifconfig eth0 0 down;
# ifrename -i eth0
wan
# ifup wan
# ifup brwan
If everything goes O.K. ifconfig will show interface eth0 renamed to wan, up but without ip address, and brwan interface holding it's previous address. Check availability of this address by ping from another host.
- At that point you can restart network by service network restart knowing that it will be back at least to wan network.
Using bridges in VE
- Now will work with bridges for VE. First we should totally disable venet by adding line DISABLED=yes to it's options file:
# echo 'DISABLED=yes' >> /etc/net/ifaces/venet0
- Let's assume Virtual ID to be 610015. Ensure /etc/vz/conf/116211.conf does not contain lines starting with IP_ADDRESS
- To create 2 VETH interfaces, eth0 and eth1, for our VE, do the following commands:
# vzctl set 160015 --netif_add eth0 --save
# vzctl set 160015 --netif_add eth1 --save
These 2 commands created in /etc/vz/conf/160015.conf the following line (wrapped here for easy reading):
NETIF="ifname=eth0,mac=00:18:51:92:67:32,\
host_ifname=veth610013.0,host_mac=00:18:51:3C:41:74;\
ifname=eth1,mac=00:18:51:01:68:5E,\
host_ifname=veth610013.1,\
host_mac=00:18:51:3D:E2:1B"
- Add the following lines to this file:
cat <<EOF >>/etc/vz/conf/610013.conf
CONFIG_CUSTOMIZED="yes"
VETH_IP_ADDRESS="10.61.0.13/24;5.88.52.183/27"
BRIDGEDEV="brlan;brwan"
VE_DEFAULT_GATEWAY="5.88.52.161"
Here I've assigned IP addresses for my interfaces eth0 and eth1 of VEID and defined corresponding HE's interfaces veth610013.{0,1}.
- Now let's create file for OpenVZ bridge configuration:
#echo <<EOF >/usr/sbin/vznetcfg.custom
#!/bin/bash
# /usr/sbin/vznetcfg.custom
# a script to bring up bridged network interfaces (veth's) in a VE
GLOBALCONFIGFILE=/etc/vz/vz.conf
VECONFIGFILE=/etc/vz/conf/$VEID.conf
vzctl=/usr/sbin/vzctl
brctl=/sbin/brctl
ip=/sbin/ip
ifconfig=/sbin/ifconfig
. $GLOBALCONFIGFILE
. $VECONFIGFILE
NETIFS=`echo $NETIF | sed 's/;/\n/g'`
for NETIFX in $NETIFS
do
NETIF_OPTIONS=`echo $NETIFX | sed 's/,/\n/g'`
for str in $NETIF_OPTIONS; do \
# getting 'ifname' parameter value
if [[ "$str" =~ "^ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VEIFNAME=${str#*=};
fi
# getting 'host_ifname' parameter value
if [[ "$str" =~ "^host_ifname=" ]]; then
# remove the parameter name from the string (along with '=')
VZHOSTIF=${str#*=};
fi
done
BRIDGEX=${BRIDGEDEV%%;*}
BRIDGEL=${BRIDGEDEV#*;}
BRIDGEDEV=$BRIDGEL;
VETH_IP_ADDRX=${VETH_IP_ADDRESS%%;*}
VETH_IP_ADDRL=${VETH_IP_ADDRESS#*;}
VETH_IP_ADDRESS=$VETH_IP_ADDRL;
if [ ! -n "$VETH_IP_ADDRX" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth IPs configured."
exit 1
fi
if [ ! -n "$VZHOSTIF" ]; then
echo "According to $CONFIGFILE VE$VEID has no veth interface configured."
exit 1
fi
if [ ! -n "$VEIFNAME" ]; then
echo "Corrupted $CONFIGFILE: no 'ifname' defined for host_ifname $VZHOSTIF."
exit 1
fi
echo "Initializing interface $VZHOSTIF for VE$VEID."
$ifconfig $VZHOSTIF 0
VEROUTEDEV=$VZHOSTIF
if [ -n "$BRIDGEX" ]; then
echo "Adding interface $VZHOSTIF to the bridge $BRIDGEX."
VEROUTEDEV=$BRIDGEX
$brctl addif $BRIDGEX $VZHOSTIF
fi
# Up the interface $VEIFNAME link in VE$VEID
$vzctl exec $VEID $ip link set $VEIFNAME up
for IP in $VETH_IP_ADDRX; do
echo "Adding an IP $IP to the $VEIFNAME for VE$VEID."
$vzctl exec $VEID $ip address add $IP dev $VEIFNAME
# removing the netmask
IP_STRIP=${IP%%/*};
echo "Adding a route from VE0 to VE$VEID."
$ip route add $IP_STRIP dev $VEROUTEDEV
done
if [ -n "$VE0_IP" ]; then
echo "Adding a route from VE$VEID to VE0."
$vzctl exec $VEID $ip route add $VE0_IP dev $VEIFNAME
fi
if [ -n "$VE_DEFAULT_GATEWAY" ]; then
echo "Setting $VE_DEFAULT_GATEWAY as a default gateway for VE$VEID."
$vzctl exec $VEID \
$ip route add default via $VE_DEFAULT_GATEWAY dev $VEIFNAME
fi
done
exit 0
EOF
It's based on This script from the Wiki on OpenVZ site, but modified to serve multiple bridges, networks and interfaces.
- Now let's configure OpenVZ to use this script:
# chmod +x /usr/sbin/vznetcfg.custom
# echo 'EXTERNAL_SCRIPT="/usr/sbin/vznetcfg.custom"' >/etc/vz/vznet.conf
At this point VE is ready to be started/restarted and should appear after start/restart in these networks.
