action #60407
closed[qac][wicked] Investigate how to get multiple NICs in different broadcast-domains when having a MM scenario
0%
Description
We discussed with Network team, that we need to have different broadcast domains when having multiple NICs.
Something like this:
Updated by cfconrad about 5 years ago
- Status changed from New to In Progress
- Assignee set to cfconrad
OpenQA offers a NETWORKS=
variable. Each network is get's it's own VLAN tag. The order is correlated to the NIC number.
e.g. NETWORKS=net1,net2
will allocate 2 VLANs and assign them to the interfaces.
@see:
openQA/lib/OpenQA/Schema/Result/Jobs.pm::prepare_for_work()
Updated by cfconrad almost 5 years ago
So it is possible to create multiple VLAN's and put each NIC in a specific one.
A "problem" might be, that they are not fully independent. As each VLAN belongs from open-vswitch perspective to the same bridge. And we have net.ipv4.ip_forward = 1
enabled on that host.
Example:¶
Bridge "br1"
Port "tap0"
Interface "0"
Port "tap1"
Interface "tap64"
Port "tap41"
Interface "tap128"
Port "br1"
Interface "br1"
type: internal
Updated by jlausuch almost 5 years ago
I would test it as follows:
1) create 2 VMs with 2 NICs in different VLANs
VM1: NIC1 (vlan X) NIC2 (vlan Y)
VM2: NIC1 (vlan X) NIC2 (vlan Y)
2) Assing 192.168.0.100/24 to NIC1 in VM1
3) Assign 192.168.0.101/24 to NIC2 in VM2
4) Check ping from VM1-NIC1 to VM2-NIC2 or viceversa
I think the net.ipv4.ip_forward = 1
option applies from br0 to other phyisical NICs in the host, but I'm not sure.
Updated by jlausuch over 4 years ago
- Project changed from 46 to openQA Tests (public)
- Subject changed from [kernel][wicked] Investigate how to get multiple NICs in different broadcast-domains when having a MM scenario to [qac][wicked] Investigate how to get multiple NICs in different broadcast-domains when having a MM scenario
- Status changed from In Progress to Workable
Updated by jlausuch over 4 years ago
- Category set to Spike/Research
- Status changed from Workable to In Progress
Updated by cfconrad over 4 years ago
I did some further tests.
Setup¶
Running two jobs (wicked_advance) in parallel each with NETWORKS=vnet1,vnet2
.
- SUT: tap43, tap107
- REF: tap42, tap106
The ovs config looks like:
14:50-autobot#[0] INTERFACES="tap43 tap107 tap42 tap106 br1"
14:52-autobot#[0] for i in $INTERFACES; do
> echo INTERFACE: $i;
> echo -n ' vlan_mode='
> ovs-vsctl get port $i vlan_mode;
> echo -n ' tag='
> ovs-vsctl get port $i tag;
> echo -n ' stp_enable='
> ovs-vsctl get bridge $i stp_enable
> done
INTERFACE: tap43
vlan_mode="dot1q-tunnel"
tag=1
stp_enable=ovs-vsctl: no row "tap43" in table Bridge
INTERFACE: tap107
vlan_mode="dot1q-tunnel"
tag=2
stp_enable=ovs-vsctl: no row "tap107" in table Bridge
INTERFACE: tap42
vlan_mode="dot1q-tunnel"
tag=1
stp_enable=ovs-vsctl: no row "tap42" in table Bridge
INTERFACE: tap106
vlan_mode="dot1q-tunnel"
tag=2
stp_enable=ovs-vsctl: no row "tap106" in table Bridge
INTERFACE: br1
vlan_mode=[]
tag=[]
stp_enable=false
09:56-autobot#[0] sysctl -a | grep ip_forward
net.ipv4.ip_forward = 1
net.ipv4.ip_forward_use_pmtu = 0
Check¶
According to https://progress.opensuse.org/issues/60407#note-3 there is no PING possible.
I also verified it with broadcast packages created with:
echo "FOO" | socat - UDP-DATAGRAM:255.255.255.255:24000,broadcast
and using tshark -i ethX -f 'port !22'
on each interface to see if the packages are visible or not.
Summary¶
We are able to build multi-host scenarios and connect individual NIC's which each other.
As an example I created 3 hosts like:
NETWORKS=vnet1,vnet2
NETWORKS=vnet2,vnet3
NETWORKS=vnet3,vnet4
and configured them to have a real routing scenario like:
eth1 eth0 eth1 eth0
+---------+ 10.6.0.0/24 +------------+ 10.7.0.0/24 +-------+
| HOST1 +<---------------->| Router |<--------------->+ HOST2 |
+---------+ |ip_forward=1| +-------+
10.6.0.2 +------------+ 10.7.0.2
10.6.0.1 10.7.0.1
Question¶
In each VLAN is the br1 present. So if an interface takes a ip from 10.0.0.0/15 it can access 10.0.2.2 (which is the default gw running NAT). Is this a problem?
Updated by cfconrad over 4 years ago
- Status changed from In Progress to Feedback
Updated by jlausuch over 4 years ago
According to the [documentation], the setup should look like this:
+-------------+
| |
| Test suite |
| (with |
| Jenkins |
| as an |
| option) |
| |
+-+---------+-+
| |
| serial, |
| ssh, +---------------+ serial, ssh,
********* | or virtio | or virtio
* Outer * | |
* World * +-+-----------+ +-+-----------+
********* | | eth0 | |
| | +-----------+ |
| eth2 | Reference | eth1 | System |
+------+ server +-----------+ Under |
| | ib0 | Tests |
| +-----------+ |
| | ... | |
+-------------+ +-------------+
Here, Outerworld would be the worker, connected with OBS bridge to REF using 10.0.2.X
Then, the other 2 interfaces connected to SUT should be different. So, do we need 3 NICs in REF?
[documentation]: https://github.com/openSUSE/wicked-testsuite/blob/master/README.setup
Updated by jlausuch over 4 years ago
Transforming this picture in our setup, it would be something like this:
┌-----------------------------------------------┓
| |
| WORKER |
| REF SUT |
| +----------+ +----------+ |
| | | | | |
| | eth0 +-----------+ eth0 | |
| | | | | |
| | eth1 +-----------+ eth1 | |
| | | | | |
| | | | | |
| | eth2 | | e̶t̶h̶2̶ | |
| +----------+ +----------+ |
| | | |
| | | |
| |_________ br0 _________| |
| | |
| |
└-----------------------------------------------┘
Updated by jlausuch over 4 years ago
- Status changed from Feedback to Closed
I created poo#67774 to follow up this topic with real implementation.