openSUSE Project Management Tool: Issueshttps://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842023-03-13T07:20:30ZopenSUSE Project Management Tool
Redmine ALP - coordination #125846 (Resolved): [epic] March Prototype testinghttps://progress.opensuse.org/issues/1258462023-03-13T07:20:30Zjlausuchjalausuch@suse.com
<p>For March prototype we have a 2 main deliverables:</p>
<ul>
<li><a href="https://confluence.suse.com/display/LEONG/01+SUSE+ALP+Bedrock" class="external">ALP Bedrock</a></li>
<li><a href="https://confluence.suse.com/display/LEONG/03+SUSE+ALP+Micro" class="external">ALP Micro</a></li>
</ul>
<p>Bedrock is like generic server-flavor OS and Micro is similar to SLE Micro or MicroOS flavor with limited set of packages.</p>
<p>QE Department has to provide test coverage for all the <a href="https://jira.suse.com/issues/?jql=project%20%3D%20PED%20AND%20fixVersion%20%3D%20%22ALP%20March%20Prototype%22" class="external">features</a> and all the images in the 2 flavors.</p>
<p>This epic aims to collect all the needed tickets related to March prototype testing.</p>
openQA Tests - action #116257 (New): [virtualization][svirt] Some workers in openqaworker2 time o...https://progress.opensuse.org/issues/1162572022-09-06T06:57:39Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-12-SP5-JeOS-for-kvm-and-xen-Updates-x86_64-jeos-extratest@svirt-xen-hvm fails in<br>
<a href="https://openqa.suse.de/tests/9459035/modules/bootloader_svirt/steps/25" class="external">bootloader_svirt</a></p>
<p>It hits the <code>MAX_JOB_TIMEOUT</code> while trying to copy the image. </p>
<p>The affected workers are:<br>
<a href="https://openqa.suse.de/admin/workers/366" class="external">openqaworker2:9</a><br>
<a href="https://openqa.suse.de/admin/workers/980" class="external">openqaworker2:10</a><br>
<a href="https://openqa.suse.de/admin/workers/1252" class="external">openqaworker2:16</a></p>
<p>Most jobs using these workers time out during this step. Other examples:<br>
<a href="https://openqa.suse.de/tests/9459036" class="external">https://openqa.suse.de/tests/9459036</a><br>
<a href="https://openqa.suse.de/tests/9459031" class="external">https://openqa.suse.de/tests/9459031</a><br>
<a href="https://openqa.suse.de/tests/9459037" class="external">https://openqa.suse.de/tests/9459037</a><br>
<a href="https://openqa.suse.de/tests/9459064" class="external">https://openqa.suse.de/tests/9459064</a><br>
<a href="https://openqa.suse.de/tests/9459069" class="external">https://openqa.suse.de/tests/9459069</a></p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/9459035" class="external">20220905-1</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/9450196" class="external">20220903-1</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=JeOS-for-kvm-and-xen-Updates&machine=svirt-xen-hvm&test=jeos-extratest&version=12-SP5" class="external">latest</a></p>
ALP - coordination #112409 (Resolved): [epic] PoC for openSUSE ALP testing in openQA https://progress.opensuse.org/issues/1124092022-06-14T15:51:37Zjlausuchjalausuch@suse.com
<p>This epic is a coordination effort for a first Proof of Concept enabling ALP testing in openqa.opensuse.org.</p>
<p>The goal is to re-use the existing tests we are running for MicroOS plus some adaptations and execute them against images.</p>
<p>The images that this PoC shall cover are:</p>
<ul>
<li>kvm-and-xen</li>
<li>kvm-and-xen_NonTransactional</li>
<li>SelfInstall</li>
<li>SelfInstall_NonTransactional</li>
</ul>
<p>Current available architecture is x86_64, but aarch64 might be available for the PoC as well.</p>
openQA Project - action #111314 (Workable): _SECRET_ variables are exposed in vars.json when the ...https://progress.opensuse.org/issues/1113142022-05-19T11:03:37Zjlausuchjalausuch@suse.com
<p>Some workers contain sensitive information using <code>_SECRET</code> variables. Those variables are hidden in the settings tab and in vars.json, as expected.<br>
However, if you restart or clone a job and cancel it while it's running, those variables are exposed in vars.json</p>
<p><img src="https://progress.opensuse.org/attachments/download/13268/vars.png" alt="" loading="lazy" /></p>
<p>NOTE: I don't want to provide links as I might give too many hints for a public place.</p>
openQA Project - action #111135 (New): Enhance email notification message content for about faile...https://progress.opensuse.org/issues/1111352022-05-16T07:57:05Zjlausuchjalausuch@suse.com
<p>The new feature introduced by <a href="https://progress.opensuse.org/issues/91605" class="external">https://progress.opensuse.org/issues/91605</a> is very useful to notify people in different ways (direct email or to Slack, which will turn into Slack message). However, those messages could be improved adding some extra information about the test name, the group name, etc.</p>
<p>This is an example of how a message in Slack looks like:<br>
<img src="https://progress.opensuse.org/attachments/download/13239/slack_message.png" alt="" loading="lazy" /></p>
<p>So, a proposal from my side could be:</p>
<pre><code>Unknown issue to be reviewed.
OpenQA test https://openqa.suse.de/tests/8762619 fails with
"Test died: script timeout: docker info at /usr/lib/os-autoinst/distribution.pm line 296."
</code></pre>
<p>I guess it's difficult to include the reason given by the failure, so something like this could be also helpful:</p>
<pre><code>Unknown issue to be reviewed.
OpenQA test https://openqa.suse.de/tests/8762619 fails in docker.
Job Group: 427 - Maintenance: Test Repo / Public Cloud Maintenance Updates
Build: 20220515-1
Flavor: AZURE-CHOST-BYOS-Updates
</code></pre>
<p>similar to when you report a poo ticket directly from the UI. </p>
openQA Tests - action #111093 (New): [containers][sporadic][s389x] test fails in boot_to_desktop ...https://progress.opensuse.org/issues/1110932022-05-13T13:22:29Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>openQA test in scenario sle-15-SP4-BCI-Updates-s390x-bci_on_SLES_15-SP2_host_docker@s390x-kvm-sle12 fails in<br>
<a href="https://openqa.suse.de/tests/8753928/modules/boot_to_desktop/steps/28" class="external">boot_to_desktop</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>The base test suite is used for job templates defined in YAML documents. It has no settings of its own.</p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/8753928" class="external">_15-SP4_10.47_minimal-image</a> (current job)</p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: <a href="https://openqa.suse.de/tests/8744352" class="external">_15-SP4_3.9_python-3.10-image</a> (or more recent)</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
<p>Always latest result in this scenario: <a href="https://openqa.suse.de/tests/latest?arch=s390x&distri=sle&flavor=BCI-Updates&machine=s390x-kvm-sle12&test=bci_on_SLES_15-SP2_host_docker&version=15-SP4" class="external">latest</a></p>
openQA Tests - action #95697 (New): [kernel][jeos][opensuse] Have a common way to add LTP reposit...https://progress.opensuse.org/issues/956972021-07-20T08:34:37Zjlausuchjalausuch@suse.com
<p>Currently, LTP tests in SLE use QA_HEAD_REPO variable<br>
<a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/kernel/install_ltp.pm" class="external">https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/master/tests/kernel/install_ltp.pm</a></p>
<pre><code> if (is_sle) {
add_qa_head_repo;
return;
}
</code></pre>
<p>Then, for openSUSE, the openSUSE tests, the condition is a bit complex:</p>
<pre><code> my $arch = '';
$arch = "_PowerPC" if is_ppc64le();
$arch = "_zSystems" if is_s390x();
$arch = ((is_x86_64 || is_aarch64) ? "Tumbleweed" : "Factory") . $arch;
$repo = "https://download.opensuse.org/repositories/benchmark:/ltp:/devel/openSUSE_$arch/";
</code></pre>
<p>and even more complex after <a href="https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/12902" class="external">this PR</a>.</p>
<p>The idea behind this ticket is to use the same function (e.g. <code>add_qa_head_repo</code>) for ALL distri/versions using a single variable (e.g. <code>QA_HEAD_REPO</code>) pointing to the repository to be used, instead of hardcoding the repository with several conditions in the code. This would affect all the kernel jobs (also for JeOS-kernel jobs) for TW and Leap in O3.</p>
openQA Tests - action #65115 (Resolved): [qac][public cloud] Storage perf test failing in ssh con...https://progress.opensuse.org/issues/651152020-04-01T06:24:17Zjlausuchjalausuch@suse.com
<p>PC tools helper VM fails to connect to the VM</p>
<p><a href="https://openqa.suse.de/tests/4070444#" class="external">https://openqa.suse.de/tests/4070444#</a><br>
<a href="https://openqa.suse.de/tests/4067508#" class="external">https://openqa.suse.de/tests/4067508#</a></p>
<pre><code># nc -vz -w 1 34.76.215.65 22; echo Hsf3L-$?-
Connection to 34.76.215.65 22 port [tcp/ssh] succeeded!
Hsf3L-0-
# cat > /tmp/scripthW6mh.sh << 'EOT_hW6mh'; echo hW6mh-$?-
> ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -i '/root/.ssh/id_rsa' "susetest@34.76.215.65" -- 'uname -r'
> EOT_hW6mh
hW6mh-0-
# echo hW6mh; bash -oe pipefail /tmp/scripthW6mh.sh ; echo SCRIPT_FINISHEDhW6mh-$?-
hW6mh
susetest@34.76.215.65: Permission denied (publickey).
</code></pre> openQA Tests - action #64797 (Resolved): [kernel][public cloud] EC2 command fails to upload imageshttps://progress.opensuse.org/issues/647972020-03-25T07:43:38Zjlausuchjalausuch@suse.com
<p>There is a new issue happening in latest build corresponding to SLES build 163.11.</p>
<p><a href="https://openqa.suse.de/tests/overview?distri=sle&version=15-SP2&build=0011&groupid=274" class="external">https://openqa.suse.de/tests/overview?distri=sle&version=15-SP2&build=0011&groupid=274</a></p>
<pre><code># ec2uploadimg --access-id 'AKIAYWW2BWC7ZK6D6TDV' -s 'LhgJISWo2A9drw67gvybqU3whwAFViXyeos3ihq6' --backing-store ssd --grub2 --machine 'x86_64' -n 'openqa-SLES15-SP2-CHOST-BYOS.x86_64-0.9.5-EC2-Build1.3.raw.xz' --virt-type hvm --sriov-support --ena-support --verbose --regions 'eu-central-1' --ssh-key-pair 'openqa1585121012_0' --private-key-file QA_SSH_KEY.pem -d 'OpenQA tests' 'SLES15-SP2-CHOST-BYOS.x86_64-0.9.5-EC2-Build1.3.raw.xz'; echo i5Gu5-$?-
Successfully created VPC with id vpc-0c8dfe70b8a7b2ffe
Successfully created internet gateway igw-07ae0d90e57169a80
Successfully created route table rtb-0b44872ab7935b9d3
Successfully created VPC subnet with id subnet-0de5c035c815caa93
Creating temporary security group
Temporary Security Group Created sg-0b495d816b0c739d1 in vpc vpc-0c8dfe70b8a7b2ffe
Successfully allowed incoming SSH port 22 for security group sg-0b495d816b0c739d1 in vpc-0c8dfe70b8a7b2ffe
Waiting for instance: i-0ae73ed595a9aba4f
. .
Waiting for volume creation: vol-08e5155f3378cefee
.
Wait for volume attachment
.
Waiting to obtain instance IP address
.
Attempt ssh connection to 18.194.37.145
. . . /root/.venv_ec2uploadimg/lib/python3.6/site-packages/paramiko/client.py:837: UserWarning: Unknown ssh-ed25519 host key for 18.194.37.145: b'fa5f17f0f6a344c0504dde11927b4cf5'
key.get_name(), hostname, hexlify(key.get_fingerprint())
An error occurred (DependencyViolation) when calling the DeleteSecurityGroup operation: resource sg-0b495d816b0c739d1 has a dependent object
</code></pre> openQA Tests - action #64724 (Resolved): [kernel][xfstests] Parted fails to create a partitionhttps://progress.opensuse.org/issues/647242020-03-22T18:31:09Zjlausuchjalausuch@suse.com
<p><a href="https://openqa.suse.de/tests/4017525#step/partition/13" class="external">https://openqa.suse.de/tests/4017525#step/partition/13</a></p>
<pre><code>parted /dev/vdb --script -- mklabel gpt
Error: partition(s) on /dev/vdb are being used.
</code></pre> openQA Tests - action #64710 (Resolved): [qac][public cloud] Failed to get credentials form Vault...https://progress.opensuse.org/issues/647102020-03-21T17:38:19Zjlausuchjalausuch@suse.com
<p>All the tests in Azure-HPC-BYOS fail to talk to Vault server. For other flavors it works: I have re-run tests at the same time in one flavor and the others and it always fails in Azure-HPC-BYOS flavor.</p>
<p><a href="https://openqa.suse.de/tests/4020950#step/run_ltp/32" class="external">https://openqa.suse.de/tests/4020950#step/run_ltp/32</a></p>
<p>Not sure what is happening in the background, but looks like when vault_get_secrets is called, it calls vault_api and fails the 3 times it calls __vault_api.</p>
<p>We should increase the debug messages here, cause it's difficult to debug with only these calls:</p>
<pre><code>[2020-03-21T18:26:42.592 CET] [debug] tests/publiccloud/run_ltp.pm:59 called publiccloud::basetest::provider_factory -> lib/publiccloud/basetest.pm:65 called publiccloud::azure::init -> lib/publiccloud/azure.pm:44 called publiccloud::azure::vault_create_credentials -> lib/publiccloud/azure.pm:64 called testapi::record_info
[2020-03-21T18:26:42.592 CET] [debug] <<< testapi::record_info(title="INFO", output="Get credentials from VAULT server.", result="ok")
[2020-03-21T18:27:12.732 CET] [debug] Maximum number of Vault request retries exceeded. Check Vault Server is up and running at /var/lib/openqa/cache/openqa.suse.de/tests/sle/lib/publiccloud/provider.pm line 549.
[2020-03-21T18:27:12.735 CET] [debug] lib/publiccloud/basetest.pm:94 called publiccloud::basetest::_cleanup -> lib/publiccloud/basetest.pm:80 called (eval) -> lib/publiccloud/basetest.pm:80 called run_ltp::cleanup -> tests/publiccloud/run_ltp.pm:103 called testapi::type_string
</code></pre> openQA Tests - action #64547 (Resolved): [kernel][public cloud] Terraform init crashes with SIGSE...https://progress.opensuse.org/issues/645472020-03-17T22:11:47Zjlausuchjalausuch@suse.com
<p>There is a new error in Terraform while running the command terraform init.<br>
Needs investigation: </p>
<p>Examples:<br>
<a href="https://openqa.suse.de/tests/3997753" class="external">https://openqa.suse.de/tests/3997753</a> (EC2)<br>
<a href="https://openqa.suse.de/tests/3997743" class="external">https://openqa.suse.de/tests/3997743</a> (GCE)<br>
and some more in Azure.</p>
<pre><code># terraform init -no-color; echo 2t4Hw-$?-
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "random" (hashicorp/random) 2.2.1...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x55c999cd8dfc]
</code></pre> openQA Tests - action #58664 (Resolved): [kernel][wicked] Needs to improve handling of situation ...https://progress.opensuse.org/issues/586642019-10-24T13:56:53Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>tcpdump commands fails</p>
<pre><code># ## START: t01_gre_tunnel_legacy
# tcpdump -s0 -U -w /tmp/tcpdumpt01_gre_tunnel_legacy.pcap >& /dev/null & export CHECK_TCPDUMP_PID=$!; echo ZUt9d-$?-
[1] 4252
ZUt9d-0-
[1]+ Exit 127 tcpdump -s0 -U -w /tmp/tcpdumpt01_gre_tunnel_legacy.pcap &> /dev/null
</code></pre>
<p><a href="https://openqa.suse.de/tests/3518286#step/t01_gre_tunnel_legacy/89" class="external">https://openqa.suse.de/tests/3518286#step/t01_gre_tunnel_legacy/89</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Maintainer: <a href="mailto:asmorodskyi@suse.de">asmorodskyi@suse.de</a> <a href="mailto:jalausuch@suse.com">jalausuch@suse.com</a> <a href="mailto:cfamullaconrad@suse.de">cfamullaconrad@suse.de</a></p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.opensuse.org/tests/1063727" class="external">20191022</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>Last good: None in QAM jobs, but it works on QA jobs. <br>
Example of expected result: <a href="https://openqa.suse.de/tests/3507684" class="external">https://openqa.suse.de/tests/3507684</a></p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>
openQA Tests - action #58268 (Resolved): [kernel][public cloud] Fix OpenQA view with different ve...https://progress.opensuse.org/issues/582682019-10-16T11:57:33Zjlausuchjalausuch@suse.com
<p>Currently, we are grouping Public Cloud tests depending on the Cloud Build and the Kiwi Build, which are most of the same different for different providers. This makes the grouping of the tests cases a bit tricky, always depending on those build numbers. This has some drawbacks (e.g. searching specific tests, JDP, ...) and makes the view very complex.</p>
<p>The idea is to come up with a solution to group the tests in a smarter way. From the ISOS POST, we could change the BUILD variable we pass to OpenQA.</p>
<p>2 ways:</p>
<p>1) Use same BUILD number as the corresponding SLES build to the PC images<br>
Advantage: everything is in the same place, we can even merge the group with SLE1X job groups</p>
<p>It would look something like this:</p>
<pre><code>Build0358 (a day ago)
Build0357 (4 days ago)
Build0350 (8 days ago)
Build0346 (8 days ago)
</code></pre>
<p>2) Group them by SLES build number and also by Provider name:<br>
Advantage: easy to find tests for specific provider<br>
Disadvantage: we can't merge it in the SLE1X groups, we would need to keep it in Public Cloud group.</p>
<p>IT would look something like this:</p>
<pre><code>Build0358-Azure (a day ago)
Build0358-EC2 (a day ago)
Build0358-GCE (a day ago)
Build0357-Azure (a day ago)
Build0357-EC2 (a day ago)
Build0357-GCE (a day ago)
...
</code></pre> openQA Tests - action #58220 (Resolved): [kernel] fadump LVM test failshttps://progress.opensuse.org/issues/582202019-10-15T20:25:44Zjlausuchjalausuch@suse.com
<a name="Observation"></a>
<h2 >Observation<a href="#Observation" class="wiki-anchor">¶</a></h2>
<p>Failure when going into grub screen<br>
<a href="https://openqa.suse.de/tests/3479900#step/kdump_and_crash/64" class="external">grub screen</a></p>
<a name="Test-suite-description"></a>
<h2 >Test suite description<a href="#Test-suite-description" class="wiki-anchor">¶</a></h2>
<p>Maintainer: Petr Cervinka <a href="mailto:pcervinka@suse.com">pcervinka@suse.com</a></p>
<a name="Reproducible"></a>
<h2 >Reproducible<a href="#Reproducible" class="wiki-anchor">¶</a></h2>
<p>Fails since (at least) Build <a href="https://openqa.suse.de/tests/3479900" class="external">0358</a><br>
But it also failed in some <a href="https://openqa.suse.de/tests/3395111" class="external">older run</a></p>
<a name="Expected-result"></a>
<h2 >Expected result<a href="#Expected-result" class="wiki-anchor">¶</a></h2>
<p>This is an example of <a href="https://openqa.suse.de/tests/3470330#step/kdump_and_crash/64" class="external">successful run</a> in the previous build.</p>
<a name="Further-details"></a>
<h2 >Further details<a href="#Further-details" class="wiki-anchor">¶</a></h2>