Project

General

Profile

action #39887

[kernel] [sle] [network] test fails in before_test - network failure

Added by pcervinka about 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Bugs in existing tests
Target version:
Start date:
2018-08-17
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

openQA test in scenario sle-15-SP1-Installer-DVD-x86_64-wicked_advanced_sut@64bit fails in
before_test

Reproducible

Fails since (at least) Build 20.3

Expected result

Last good: (unknown) (or more recent)

Further details

Always latest result in this scenario: latest

History

#1 Updated by pcervinka about 3 years ago

  • Subject changed from [kernel-qa] [sle] [network] test fails in before_test - network failure to [kernel] [sle] [network] test fails in before_test - network failure

#2 Updated by pcervinka about 3 years ago

Reproduction links without any modification in code:
15SP1 http://10.100.12.105/tests/446#step/before_test/19
12SP4 http://10.100.12.105/tests/447#step/before_test/19

It is clearly visible that test fails during openvpn installation.

#3 Updated by pcervinka about 3 years ago

SLE12SP4 installation use only mounted iso, there are no network repositories:
http://10.100.12.105/tests/449#step/before_test/23

# | Alias             | Name              | Enabled | GPG Check | Refresh | URI                                                      
 --+-------------------+-------------------+---------+-----------+---------+----------------------------------------------------------
 1 | SLES12-SP4-12.4-0 | SLES12-SP4-12.4-0 | Yes     | (r ) Yes  | No      | cd:///?devices=/dev/disk/by-id/scsi-0QEMU_QEMU_CD-ROM_cd0
 98LF4-0-

openvpn is in "CD" repo":
http://10.100.12.105/tests/449#step/before_test/25

 Information for package openvpn:
 --------------------------------
 Repository     : SLES12-SP4-12.4-0                                       
 Name           : openvpn                                                 
 Version        : 2.3.8-16.20.1         

This is the reason why openvpn pass on 12SP4 even without network.

SLE15SP1 has many repositories defined, so it really fails if the network is not configured:
http://10.100.12.105/tests/448#step/before_test/23

Repository priorities are without effect. All enabled repositories share the same priority.

#  | Alias                                                                                           | Name                                                   | Enabled | GPG Check | Refresh | URI                                                                                                  
---+-------------------------------------------------------------------------------------------------+--------------------------------------------------------+---------+-----------+---------+------------------------------------------------------------------------------------------------------
 1 | Basesystem_Module_15_SP1_x86_64:SLE-Module-Basesystem15-SP1-Debuginfo-Pool                      | SLE-Module-Basesystem15-SP1-Debuginfo-Pool             | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build21.2-Media1/         
 2 | Basesystem_Module_15_SP1_x86_64:SLE-Module-Basesystem15-SP1-Debuginfo-Updates                   | SLE-Module-Basesystem15-SP1-Debuginfo-Updates          | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build21.2-Media1/         
 3 | Basesystem_Module_15_SP1_x86_64:SLE-Module-Basesystem15-SP1-Pool                                | SLE-Module-Basesystem15-SP1-Pool                       | Yes     | (r ) Yes  | No      | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build21.2-Media1/         
 4 | Basesystem_Module_15_SP1_x86_64:SLE-Module-Basesystem15-SP1-Source-Pool                         | SLE-Module-Basesystem15-SP1-Source-Pool                | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build21.2-Media1/         
 5 | Basesystem_Module_15_SP1_x86_64:SLE-Module-Basesystem15-SP1-Updates                             | SLE-Module-Basesystem15-SP1-Updates                    | Yes     | (r ) Yes  | Yes     | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Basesystem-POOL-x86_64-Build21.2-Media1/         
 6 | SLES15-SP1-15.1-0                                                                               | SLES15-SP1-15.1-0                                      | No      | ----      | ----    | cd:///?devices=/dev/disk/by-id/scsi-0QEMU_QEMU_CD-ROM_cd0                                            
 7 | SUSE_Linux_Enterprise_Server_15_SP1_x86_64:SLE-Product-SLES15-SP1-Debuginfo-Pool                | SLE-Product-SLES15-SP1-Debuginfo-Pool                  | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Product-SLES-POOL-x86_64-Build21.2-Media1/              
 8 | SUSE_Linux_Enterprise_Server_15_SP1_x86_64:SLE-Product-SLES15-SP1-Debuginfo-Updates             | SLE-Product-SLES15-SP1-Debuginfo-Updates               | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Product-SLES-POOL-x86_64-Build21.2-Media1/              
 9 | SUSE_Linux_Enterprise_Server_15_SP1_x86_64:SLE-Product-SLES15-SP1-Pool                          | SLE-Product-SLES15-SP1-Pool                            | Yes     | (r ) Yes  | No      | http://openqa.suse.de/assets/repo/SLE-15-SP1-Product-SLES-POOL-x86_64-Build21.2-Media1/              
10 | SUSE_Linux_Enterprise_Server_15_SP1_x86_64:SLE-Product-SLES15-SP1-Source-Pool                   | SLE-Product-SLES15-SP1-Source-Pool                     | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Product-SLES-POOL-x86_64-Build21.2-Media1/              
11 | SUSE_Linux_Enterprise_Server_15_SP1_x86_64:SLE-Product-SLES15-SP1-Updates                       | SLE-Product-SLES15-SP1-Updates                         | Yes     | (r ) Yes  | Yes     | http://openqa.suse.de/assets/repo/SLE-15-SP1-Product-SLES-POOL-x86_64-Build21.2-Media1/              
12 | Server_Applications_Module_15_SP1_x86_64:SLE-Module-Server-Applications15-SP1-Debuginfo-Pool    | SLE-Module-Server-Applications15-SP1-Debuginfo-Pool    | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Server-Applications-POOL-x86_64-Build21.2-Media1/
13 | Server_Applications_Module_15_SP1_x86_64:SLE-Module-Server-Applications15-SP1-Debuginfo-Updates | SLE-Module-Server-Applications15-SP1-Debuginfo-Updates | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Server-Applications-POOL-x86_64-Build21.2-Media1/
14 | Server_Applications_Module_15_SP1_x86_64:SLE-Module-Server-Applications15-SP1-Pool              | SLE-Module-Server-Applications15-SP1-Pool              | Yes     | (r ) Yes  | No      | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Server-Applications-POOL-x86_64-Build21.2-Media1/
15 | Server_Applications_Module_15_SP1_x86_64:SLE-Module-Server-Applications15-SP1-Source-Pool       | SLE-Module-Server-Applications15-SP1-Source-Pool       | No      | ----      | ----    | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Server-Applications-POOL-x86_64-Build21.2-Media1/
16 | Server_Applications_Module_15_SP1_x86_64:SLE-Module-Server-Applications15-SP1-Updates           | SLE-Module-Server-Applications15-SP1-Updates           | Yes     | (r ) Yes  | Yes     | http://openqa.suse.de/assets/repo/SLE-15-SP1-Module-Server-Applications-POOL-x86_64-Build21.2-Media1/
98LF4-0-
# zypper info openvpn ; echo ~MKCc-$?-
Refreshing service 'Basesystem_Module_15_SP1_x86_64'.
Problem retrieving the repository index file for service 'Basesystem_Module_15_SP1_x86_64':
Download (curl) error for 'http://all-21.2.proxy.scc.suse.de/access/services/1757/repo/repoindex.xml?cookies=0&credentials=Basesystem_Module_15_SP1_x86_64':
Error code: Connection failed
Error message: Could not resolve host: all-21.2.proxy.scc.suse.de

This looks like test logic failure, which silently works on SLE12SP4.

#4 Updated by pcervinka about 3 years ago

If the openvpn installation is moved to the end of the test, it still
fails on 15SP1, but at different place:
http://10.100.12.105/tests/450#step/before_test/25

 # Result:
  systemctl --no-pager is-active network ; echo 6ChHn-$?-
 inactive
 6ChHn-3-

12SP4: http://10.100.12.105/tests/451#step/before_test/25

# wait_serial expected: qr/6ChHn-\d+-/

 # Result:
  systemctl --no-pager is-active network ; echo 6ChHn-$?-
 active
 6ChHn-0-

There is difference in systemctl --no-pager is-active network between 15SP1 and 12P4.
12SP4 reports network active and 15SP1 not.

#5 Updated by pcervinka about 3 years ago

Cloned several jobs for 15SP1
http://10.100.12.105/tests/459
.
.
http://10.100.12.105/tests/478

All failed during systemctl is-active network.

It looks like is-active test is run to early, if there is inserted simple check like
systemctl status network, is-active test is ok:

     script_run("systemctl status network");
     systemctl('is-active network');

http://10.100.12.105/tests/492
.
.
http://10.100.12.105/tests/513

But there is still another failure later during test run which needs to be fixed.

#6 Updated by pcervinka about 3 years ago

DNS needs to be properly configured to don't fail. Did a simple POC:

     script_run('echo "nameserver 10.100.2.8" >> /etc/resolv.conf'); 
     script_run('echo "nameserver 10.100.2.10" >> /etc/resolv.conf'); 

And before_test works fine:
http://10.100.12.105/tests/536#step/before_test/82

Next step is to prepare PR with fixes.

#7 Updated by pcervinka about 3 years ago

Summary to fix:

  1. Installation of openvpn needs to be done after is network configured
    The difference between SLE12SP4 and SLE15P1 is in repositories, SLE12SP4 has only
    repo from installation "CDROM" and no network repositories. SLE15SP1 has network repositories
    and it basically failed without network
    TODO: move openvpn installation to the end of the test

  2. systemctl is-active network is run too early after system boot, system needs more time to
    start all systemd services
    TODO: add wait or more tries to is-active test

  3. DNS must be configured in configuration
    TODO: add DNS configuration to manual network configuration

#8 Updated by pcervinka about 3 years ago

Issue was discussed with pvorel, we agreed to wait for Anton to check final solution.

#10 Updated by pcervinka about 3 years ago

Wicked advanced is OK now:
https://openqa.suse.de/tests/1974956
https://openqa.suse.de/tests/1974954

But Wicked basic doesn't work.

#11 Updated by pcervinka about 3 years ago

  • Status changed from In Progress to Resolved

Wicked basic issues is not related to our change, it didn't work even before. Basic is covered in
https://progress.opensuse.org/issues/40181.

#12 Updated by sebchlad about 3 years ago

  • Target version changed from 445 to Done

Also available in: Atom PDF