action #60407
[qac][wicked] Investigate how to get multiple NICs in different broadcast-domains when having a MM scenario
0%
Description
We discussed with Network team, that we need to have different broadcast domains when having multiple NICs.
Something like this:
History
#1
Updated by cfconrad over 3 years ago
- Status changed from New to In Progress
- Assignee set to cfconrad
OpenQA offers a NETWORKS=
variable. Each network is get's it's own VLAN tag. The order is correlated to the NIC number.
e.g. NETWORKS=net1,net2
will allocate 2 VLANs and assign them to the interfaces.
@see:
openQA/lib/OpenQA/Schema/Result/Jobs.pm::prepare_for_work()
#2
Updated by cfconrad over 3 years ago
So it is possible to create multiple VLAN's and put each NIC in a specific one.
A "problem" might be, that they are not fully independent. As each VLAN belongs from open-vswitch perspective to the same bridge. And we have net.ipv4.ip_forward = 1
enabled on that host.
Example:¶
Bridge "br1" Port "tap0" Interface "0" Port "tap1" Interface "tap64" Port "tap41" Interface "tap128" Port "br1" Interface "br1" type: internal
#3
Updated by jlausuch over 3 years ago
I would test it as follows:
1) create 2 VMs with 2 NICs in different VLANs
VM1: NIC1 (vlan X) NIC2 (vlan Y)
VM2: NIC1 (vlan X) NIC2 (vlan Y)
2) Assing 192.168.0.100/24 to NIC1 in VM1
3) Assign 192.168.0.101/24 to NIC2 in VM2
4) Check ping from VM1-NIC1 to VM2-NIC2 or viceversa
I think the net.ipv4.ip_forward = 1
option applies from br0 to other phyisical NICs in the host, but I'm not sure.
#4
Updated by jlausuch about 3 years ago
- Project changed from SUSE QA to openQA Tests
- Subject changed from [kernel][wicked] Investigate how to get multiple NICs in different broadcast-domains when having a MM scenario to [qac][wicked] Investigate how to get multiple NICs in different broadcast-domains when having a MM scenario
- Status changed from In Progress to Workable
#5
Updated by jlausuch about 3 years ago
- Category set to Spike/Research
- Status changed from Workable to In Progress
#6
Updated by cfconrad about 3 years ago
I did some further tests.
Setup¶
Running two jobs (wicked_advance) in parallel each with NETWORKS=vnet1,vnet2
.
- SUT: tap43, tap107
- REF: tap42, tap106
The ovs config looks like:
14:50-autobot#[0] INTERFACES="tap43 tap107 tap42 tap106 br1" 14:52-autobot#[0] for i in $INTERFACES; do > echo INTERFACE: $i; > echo -n ' vlan_mode=' > ovs-vsctl get port $i vlan_mode; > echo -n ' tag=' > ovs-vsctl get port $i tag; > echo -n ' stp_enable=' > ovs-vsctl get bridge $i stp_enable > done INTERFACE: tap43 vlan_mode="dot1q-tunnel" tag=1 stp_enable=ovs-vsctl: no row "tap43" in table Bridge INTERFACE: tap107 vlan_mode="dot1q-tunnel" tag=2 stp_enable=ovs-vsctl: no row "tap107" in table Bridge INTERFACE: tap42 vlan_mode="dot1q-tunnel" tag=1 stp_enable=ovs-vsctl: no row "tap42" in table Bridge INTERFACE: tap106 vlan_mode="dot1q-tunnel" tag=2 stp_enable=ovs-vsctl: no row "tap106" in table Bridge INTERFACE: br1 vlan_mode=[] tag=[] stp_enable=false 09:56-autobot#[0] sysctl -a | grep ip_forward net.ipv4.ip_forward = 1 net.ipv4.ip_forward_use_pmtu = 0
Check¶
According to https://progress.opensuse.org/issues/60407#note-3 there is no PING possible.
I also verified it with broadcast packages created with:
echo "FOO" | socat - UDP-DATAGRAM:255.255.255.255:24000,broadcast
and using tshark -i ethX -f 'port !22'
on each interface to see if the packages are visible or not.
Summary¶
We are able to build multi-host scenarios and connect individual NIC's which each other.
As an example I created 3 hosts like:
NETWORKS=vnet1,vnet2
NETWORKS=vnet2,vnet3
NETWORKS=vnet3,vnet4
and configured them to have a real routing scenario like:
eth1 eth0 eth1 eth0 +---------+ 10.6.0.0/24 +------------+ 10.7.0.0/24 +-------+ | HOST1 +<---------------->| Router |<--------------->+ HOST2 | +---------+ |ip_forward=1| +-------+ 10.6.0.2 +------------+ 10.7.0.2 10.6.0.1 10.7.0.1
Question¶
In each VLAN is the br1 present. So if an interface takes a ip from 10.0.0.0/15 it can access 10.0.2.2 (which is the default gw running NAT). Is this a problem?
#7
Updated by cfconrad about 3 years ago
- Status changed from In Progress to Feedback
#8
Updated by jlausuch about 3 years ago
According to the documentation, the setup should look like this:
+-------------+ | | | Test suite | | (with | | Jenkins | | as an | | option) | | | +-+---------+-+ | | | serial, | | ssh, +---------------+ serial, ssh, ********* | or virtio | or virtio * Outer * | | * World * +-+-----------+ +-+-----------+ ********* | | eth0 | | | | +-----------+ | | eth2 | Reference | eth1 | System | +------+ server +-----------+ Under | | | ib0 | Tests | | +-----------+ | | | ... | | +-------------+ +-------------+
Here, Outerworld would be the worker, connected with OBS bridge to REF using 10.0.2.X
Then, the other 2 interfaces connected to SUT should be different. So, do we need 3 NICs in REF?
#9
Updated by jlausuch about 3 years ago
Transforming this picture in our setup, it would be something like this:
┌-----------------------------------------------┓ | | | WORKER | | REF SUT | | +----------+ +----------+ | | | | | | | | | eth0 +-----------+ eth0 | | | | | | | | | | eth1 +-----------+ eth1 | | | | | | | | | | | | | | | | eth2 | | e̶t̶h̶2̶ | | | +----------+ +----------+ | | | | | | | | | | |_________ br0 _________| | | | | | | └-----------------------------------------------┘
#10
Updated by jlausuch about 3 years ago
- Status changed from Feedback to Closed
I created poo#67774 to follow up this topic with real implementation.