Project

General

Profile

action #158104

Updated by okurz about 1 month ago

## Observation 

 openQA test in scenario sle-15-SP6-Online-ppc64le-ha_beta_supportserver@ppc64le-2g fails in 
 [setup](https://openqa.suse.de/tests/13885455/modules/setup/steps/84) 

 https://openqa.suse.de/tests/13885455#step/setup/84    (see attachment p1.png) 

 https://openqa.suse.de/tests/13885471#step/setup/30 (see attachment p2.png)    It missed "$" before "?". 

 https://openqa.suse.de/tests/13885404#step/setup/12 (see attachment p3.png) 

 https://openqa.suse.de/tests/13885407#step/setup/9 (see attachment p4.png) 

 I think this may related with the high work load of underlying ppc64 worker. 

 All on "mania" 

 ## Test suite description 
 The base test suite is used for job templates defined in YAML documents. It has no settings of its own. 


 ## Reproducible 

 Fails since (at least) Build [73.1](https://openqa.suse.de/tests/13885455) (current job) 


 ## Expected result 

 Last good: [67.1](https://openqa.suse.de/tests/13829359) (or more recent) 

 


 ## Suggestions 
 * Identify the affected machines and workers, apply mitigations to prevent recurring typing issues, e.g. reducing CPU load 
 * Restart related failed jobs 
 * Identify follow-up tasks 
 * Reduce the number of worker instances as a first mitigation measure. https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/759 (merged) 
 * Make the alert for CPU load more strict - #158113 
 * Evaluate the impact on video encoding in particular on ppc64le, maybe ffmpeg on Power8 kvm is inefficient - #158116 
 * Check existing ffmpeg processes on mania which take a lot of CPU time - #158116 

 ## Out of scope 
 * ffmpeg impact investigation -> #158113 
 * code improvements -> #158125 
 * improving the alert -> #158113 

 ## Further details 

 Always latest result in this scenario: [latest](https://openqa.suse.de/tests/latest?arch=ppc64le&distri=sle&flavor=Online&machine=ppc64le-2g&test=ha_beta_supportserver&version=15-SP6) 


 

Back