https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842020-10-09T10:04:33ZopenSUSE Project Management ToolopenQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3327192020-10-09T10:04:33Zokurzokurz@suse.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/332719/diff?detail_id=328120">diff</a>)</li></ul> openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3328932020-10-10T09:08:02Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> set to <i>2020-10-13</i></li><li><strong>Status</strong> changed from <i>Workable</i> to <i>Feedback</i></li><li><strong>Assignee</strong> set to <i>okurz</i></li></ul><p>At first I realized that openqa-aarch64 has not been upgrade since 11 days. It sems that the repo <a href="http://download.opensuse.org/repositories/Virtualization/openSUSE_Factory_ARM/">http://download.opensuse.org/repositories/Virtualization/openSUSE_Factory_ARM/</a> should now be <br>
<a href="http://download.opensuse.org/repositories/Virtualization/openSUSE_Factory/">http://download.opensuse.org/repositories/Virtualization/openSUSE_Factory/</a> instead. Did <code>transactional-upgrade dup reboot</code> and then in transactional-shell following <a href="https://progress.opensuse.org/projects/openqav3/wiki/#Distribution-upgrades">https://progress.opensuse.org/projects/openqav3/wiki/#Distribution-upgrades</a>.</p>
<p>All good so far, machine is back up, no failed services, all worker instances currently working on jobs. What I have seen in SoL:</p>
<pre><code>Welcome to openSUSE Leap 15.2 - Kernel 5.3.18-lp152.44-default (ttyAMA0).
openqa-aarch64 login: [ 120.648267] watchdog: BUG: soft lockup - CPU#6 stuck for 22s! [qemu-system-aar:5851]
[ 120.668315] watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [qemu-system-aar:4798]
[ 120.678321] watchdog: BUG: soft lockup - CPU#15 stuck for 22s! [qemu-system-aar:4354]
[ 120.698359] watchdog: BUG: soft lockup - CPU#22 stuck for 22s! [qemu-system-aar:5452]
[ 120.699337] watchdog: BUG: soft lockup - CPU#23 stuck for 22s! [qemu-system-aar:5436]
[ 120.728308] watchdog: BUG: soft lockup - CPU#30 stuck for 22s! [qemu-system-aar:5875]
[ 120.798340] watchdog: BUG: soft lockup - CPU#51 stuck for 22s! [qemu-system-aar:4128]
[ 124.678636] watchdog: BUG: soft lockup - CPU#14 stuck for 23s! [qemu-system-aar:5349]
[ 124.698641] watchdog: BUG: soft lockup - CPU#21 stuck for 23s! [qemu-system-aar:5373]
[ 124.718629] watchdog: BUG: soft lockup - CPU#28 stuck for 23s! [qemu-system-aar:5922]
[ 124.798631] watchdog: BUG: soft lockup - CPU#53 stuck for 23s! [migration/53:277]
[ 124.808628] watchdog: BUG: soft lockup - CPU#54 stuck for 23s! [qemu-system-aar:5908]
[ 144.688583] watchdog: BUG: soft lockup - CPU#20 stuck for 21s! [qemu-system-aar:5910]
[ 144.748542] watchdog: BUG: soft lockup - CPU#38 stuck for 22s! [migration/38:202]
[ 144.778539] watchdog: BUG: soft lockup - CPU#46 stuck for 22s! [migration/46:242]
[ 144.788539] watchdog: BUG: soft lockup - CPU#50 stuck for 22s! [migration/50:262]
[ 144.788542] watchdog: BUG: soft lockup - CPU#48 stuck for 22s! [migration/48:252]
[ 144.808538] watchdog: BUG: soft lockup - CPU#56 stuck for 22s! [migration/56:292]
[ 148.748527] watchdog: BUG: soft lockup - CPU#37 stuck for 22s! [migration/37:197]
[ME] Fault detect start!
[ME] Fault detect start!
[ME] Ipmi data check failed!
</code></pre>
<p>From the timer one can see that this was 120s after bootup.</p>
<p>So far jobs seem to be fine, e.g. <a href="https://openqa.opensuse.org/tests/1427176">https://openqa.opensuse.org/tests/1427176</a></p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3328992020-10-10T13:30:51Zokurzokurz@suse.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Resolved</i></li></ul><p>other jobs have returned similar results to in before as well so I regard the upgrade to be finished successfully.</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3329052020-10-10T14:56:17Zokurzokurz@suse.com
<ul><li><strong>Copied to</strong> <i><a class="issue tracker-4 status-3 priority-5 priority-high3 closed child" href="/issues/73189">action #73189</a>: Upgrade o3 workers to openSUSE Leap 15.2 after openqa-aarch64 already done</i> added</li></ul> openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3331422020-10-12T09:17:00Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>Thanks for the upgrade to Leap 15.2.<br>
I noticed that some mistyping errors are occurring again, whereas they were nearly absent on this machine before the upgrade.</p>
<ul>
<li><a href="https://openqa.opensuse.org/tests/1429694#step/xterm/5" class="external">https://openqa.opensuse.org/tests/1429694#step/xterm/5</a></li>
<li><a href="https://openqa.opensuse.org/tests/1430080#step/firstrun/45" class="external">https://openqa.opensuse.org/tests/1430080#step/firstrun/45</a></li>
<li><a href="https://openqa.opensuse.org/tests/1429876#step/php7/2" class="external">https://openqa.opensuse.org/tests/1429876#step/php7/2</a></li>
</ul>
<p>This is probably related to the <code>CPU stuck</code> messages.</p>
<p>qemu version before: 3.1.1.1<br>
qemu version now: 4.2.1</p>
<p>We should also check the SSD which could be "busy" or slower for some reason.</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3331512020-10-12T09:34:10Zggardet_armguillaume.gardet@arm.com
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>In Progress</i></li></ul> openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3331752020-10-12T09:56:30Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>There are also few corrupted screen cases: <a href="https://openqa.opensuse.org/tests/1429734#step/welcome/12" class="external">https://openqa.opensuse.org/tests/1429734#step/welcome/12</a></p>
<p>and <code>qemu-img create</code> failures: <a href="https://openqa.opensuse.org/tests/1429051" class="external">https://openqa.opensuse.org/tests/1429051</a></p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3332382020-10-12T11:13:00Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> changed from <i>2020-10-13</i> to <i>2020-10-16</i></li><li><strong>Assignee</strong> changed from <i>okurz</i> to <i>ggardet_arm</i></li></ul><p>I provided ssh access to the machine for ggardet_arm so that he/you can check yourself. What we could do is 1. rollback to the previous state of 15.1, 2. install a qemu version from the Virtualization repo instead, 3. debug manually on the host, 4. reduce worker instance numbers and monitor the overall load or stress on the system. I would appreciate if you can lead this effort and I just support :)</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3342282020-10-12T13:42:31Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>There is also API failures: <a href="https://openqa.opensuse.org/tests/1429899/file/worker-log.txt" class="external">https://openqa.opensuse.org/tests/1429899/file/worker-log.txt</a></p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3351912020-10-12T14:51:50Zlivdywanliv.dywan@suse.com
<ul></ul><p>ggardet_arm wrote:</p>
<blockquote>
<p>There is also API failures: <a href="https://openqa.opensuse.org/tests/1429899/file/worker-log.txt" class="external">https://openqa.opensuse.org/tests/1429899/file/worker-log.txt</a></p>
</blockquote>
<p>I guess you mean these? Looks like DNS issues 🤔</p>
<blockquote>
<p>[2020-10-12T15:06:13.0627 CEST] [error] Uploading artefact textinfo-29.png failed: Can't connect: Temporary failure in name resolution<br>
[...]<br>
[2020-10-12T15:37:01.0593 CEST] [error] Error uploading textinfo-systemctl_status.log: connection error: Can't connect: Temporary failure in name resolution</p>
</blockquote>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3352002020-10-12T15:51:46Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>WebUI claims: <code>Reason: api failure</code></p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3356052020-10-13T11:21:56Zokurzokurz@suse.com
<ul><li><strong>Subject</strong> changed from <i>Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor)</i> to <i>Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"backend died: Migrate to file failed"</i></li></ul><p>I also found <a href="https://openqa.opensuse.org/tests/1430888" class="external">https://openqa.opensuse.org/tests/1430888</a> with reason "Reason: backend died: Migrate to file failed, it has been running for more than 240 seconds at /usr/lib/os-autoinst/backend/qemu.pm line 267." pointing to performance problems.</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3356082020-10-13T11:25:15Zokurzokurz@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-2 priority-3 priority-lowest" href="/issues/73297">action #73297</a>: auto_review:"(?s)Running on openqa-aarch64:.*considering VNC stalled.*THERE IS NOTHING TO READ"</i> added</li></ul> openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3356562020-10-13T12:55:14Zokurzokurz@suse.com
<ul></ul><p>ggardet_arm asked me to debug boot problems over IPMI which is unfortunately only accessible SUSE-internal. <code>sol activate</code> does show no output so did <code>power reset</code> while <code>sol activate</code> is active in another console. Reached boot menu and confirming default indeed yields problems. Fails to boot 15.1, no helpful message, nothing over serial. <code>power reset</code> again and pressing buttons to abort grub menu timeout, e.g. "esc" and "up". Snapshot menu takes some seconds to load but is there and shows many entries so that should be ok. First selecting last good boot, i.e. "* openSUSE Leap 15.2 (,2020-10-13T11:56,post,yast snapper)".</p>
<p>Messages in sol:</p>
<pre><code>Loading Linux 5.3.18-lp152.44-default ...
Loading initial ramdisk ...
CoreGetMemoryMap: MapKey(3EDBA3E0) = D028, mMemoryMapKey(3EDE4B98) = D028.
CoreGetMemoryMap: MapKey(3EDBA400) = D028, mMemoryMapKey(3EDE4B98) = D028.
Loading driver at 0x0002C261000 EntryPoint=0x0002D25751C
Loading driver at 0x0002C261000 EntryPoint=0x0002D25751C
CoreGetMemoryMap: MapKey(3EDBA3A8) = D02C, mMemoryMapKey(3EDE4B98) = D02C.
CoreGetMemoryMap: MapKey(3EDBA3A8) = D02D, mMemoryMapKey(3EDE4B98) = D02D.
CoreGetMemoryMap: MapKey(3EDBA358) = D02E, mMemoryMapKey(3EDE4B98) = D02E.
CoreGetMemoryMap: MapKey(3EDBA358) = D02F, mMemoryMapKey(3EDE4B98) = D02F.
EFI stub: Booting Linux Kernel...
EFI stub: EFI_RNG_PROTOCOL unavailable, no randomness supplied
EFI stub: Using DTB from configuration table
CoreGetMemoryMap: MapKey(3EDBA348) = D148, mMemoryMapKey(3EDE4B98) = D148.
CoreGetMemoryMap: MapKey(3EDBA348) = D149, mMemoryMapKey(3EDE4B98) = D149.
EFI stub: Exiting boot services and installing virtual address map...
CoreGetMemoryMap: MapKey(3EDBA298) = D1D3, mMemoryMapKey(3EDE4B98) = D1D3.
CoreGetMemoryMap: MapKey(3EDBA298) = D1D4, mMemoryMapKey(3EDE4B98) = D1D4.
CoreGetMemoryMap: MapKey(3EDBA348) = D1D6, mMemoryMapKey(3EDE4B98) = D1D6.
CoreGetMemoryMap: MapKey(3EDBA348) = D1D7, mMemoryMapKey(3EDE4B98) = D1D7.
CoreGetMemoryMap: MapKey(3EDBA308) = D1D7, mMemoryMapKey(3EDE4B98) = D1D7.
CoreGetMemoryMap: MapKey(3EDBA308) = D1D8, mMemoryMapKey(3EDE4B98) = D1D8.
[SAL_ClearAffiliationSMP,324]it is going to hard reset dev:0x500E004AAAAAAA1F
SAS ExitBootServicesEvent
GMAC ExitBootServicesEvent
GMAC ExitBootServicesEvent
OHCI ExitBootServicesEvent
IPMI ExitBootService Event
M3 sram flag set.........done!IPMI ExitBootServicesEvent
sEe c
u[MEi ii]ic
[MdiSMMU ExitBootServicesEvent
ab[Md icc ensbl
dMsucc2s .n
e[ Eu cecs.nab bMe] iuc dis.
[lE] cocteoll
t[su] eonfrollru0 esrfuliy
t[MEc eontrolls 1 easeu ly
t[sEc eontuolyer M ]arof ioil ruc esrfu ly
successfully
[ME] dimm 0 present
[ME] dimm 0 present
SetOSWatchdog CountDown 3000. Status Success
BIOS Is Over! Bye~
[ME] dimm 1 present
[ME] dimm 1 present
[ME] dimm 2 present
[ME] dimm 2 present
[ME] dimm 3 present
[ME] dimm 3 present
[ME] dimm 4 present
[ME] dimm 4 present
[ME] dimm 5 present
[ME] dimm 5 present
[ME] dimm 6 present
[ME] dimm 6 present
[ME] dimm 7 present
[ME] dimm 7 present
</code></pre>
<p>and after some seconds booting kernel. Successfully booted and triggered reboot to crosscheck if I can find a bootable Leap 15.1 variant.</p>
<p>Selected "openSUSE Leap 15.1 (,2020-10-10T07:42,Snapshot Update of <a class="issue tracker-4 status-5 priority-6 priority-high2 closed parent behind-schedule" title="action: www.opensuse.org (Closed)" href="https://progress.opensuse.org/issues/156">#156</a>)" and I get:</p>
<pre><code> Booting `openSUSE Leap 15.1 '
Loading Linux 4.12.14-lp151.28.67-default ...
Loading initial ramdisk ...
CoreGetMemoryMap: MapKey(3EDBA410) = 4D1E, mMemoryMapKey(3EDE4B98) = 4D1E.
CoreGetMemoryMap: MapKey(3EDBA430) = 4D1E, mMemoryMapKey(3EDE4B98) = 4D1E.
Loading driver at 0x0002D596000 EntryPoint=0x0002E30BB10
Loading driver at 0x0002D596000 EntryPoint=0x0002E30BB10
CoreGetMemoryMap: MapKey(3EDBA3E8) = 4D22, mMemoryMapKey(3EDE4B98) = 4D22.
CoreGetMemoryMap: MapKey(3EDBA3E8) = 4D23, mMemoryMapKey(3EDE4B98) = 4D23.
CoreGetMemoryMap: MapKey(3EDBA398) = 4D24, mMemoryMapKey(3EDE4B98) = 4D24.
CoreGetMemoryMap: MapKey(3EDBA398) = 4D25, mMemoryMapKey(3EDE4B98) = 4D25.
SmiGraphicsOutputQueryMode +
SmiGraphicsOutputQueryMode -
SmiGraphicsOutputQueryMode +
SmiGraphicsOutputQueryMode -
CoreGetMemoryMap: MapKey(3EDBA398) = 4D28, mMemoryMapKey(3EDE4B98) = 4D28.
CoreGetMemoryMap: MapKey(3EDBA398) = 4D29, mMemoryMapKey(3EDE4B98) = 4D29.
CoreGetMemoryMap: MapKey(3EDBA2E8) = 4D29, mMemoryMapKey(3EDE4B98) = 4D29.
CoreGetMemoryMap: MapKey(3EDBA2E8) = 4D2A, mMemoryMapKey(3EDE4B98) = 4D2A.
CoreGetMemoryMap: MapKey(3EDBA398) = 4D2C, mMemoryMapKey(3EDE4B98) = 4D2C.
CoreGetMemoryMap: MapKey(3EDBA398) = 4D2D, mMemoryMapKey(3EDE4B98) = 4D2D.
CoreGetMemoryMap: MapKey(3EDBA358) = 4D2D, mMemoryMapKey(3EDE4B98) = 4D2D.
CoreGetMemoryMap: MapKey(3EDBA358) = 4D2E, mMemoryMapKey(3EDE4B98) = 4D2E.
[SAL_ClearAffiliationSMP,324]it is going to hard reset dev:0x500E004AAAAAAA1F
SAS ExitBootServicesEvent
GMAC ExitBootServicesEvent
GMAC ExitBootServicesEvent
OHCI ExitBootServicesEvent
IPMI ExitBootService Event
M3 sram flag set.........done!IPMI ExitBootServicesEvent
p
_ME]t
c dEs b2e SMMU ExitBootServicesEvent
si2
MEa lec sncbees
c dEsablcddsscccess.uc[Me] i2c en]bied suncbss.
[cE] sont[ollec 0 raree nit suc enstuslc
sMfullontrMEl ron raref in t succesif suy
[ME] lon
roE]ec 2 talee nit succissfullyc
[ME]llont[oll r 3ttolee nitasec inst lucc
ssfully
[ME] dimm 0 present
[ME] dimm 0 present
SetOSWatchdog CountDown 3000. Status Success
BIOS Is Over! Bye~
[ME] dimm 1 present
[ME] dimm 1 present
[ME] dimm 2 present
[ME] dimm 2 present
[ME] dimm 3 present
[ME] dimm 3 present
[ME] dimm 4 present
[ME] dimm 4 present
[ME] dimm 5 present
[ME] dimm 5 present
[ME] dimm 6 present
[ME] dimm 6 present
[ME] dimm 7 present
[ME] dimm 7 present
</code></pre>
<p>but then nothing (for at least a minute). Trying the boot menu one before that, that is nr. <code>#155</code>. No luck either. Back to <code>#178</code> on openSUSE Leap 15.2 . I suggest to keep Leap 15.2 for now but try with different qemu versions. Also the current default boot entry is still <code>#156</code> so beware to activate <code>#178</code> or similar before rebooting. Maybe a new snapshot created with <code>transactional-update</code> would automatically do that anyway.</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3356952020-10-13T16:18:10Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>I created <a href="https://bugzilla.opensuse.org/show_bug.cgi?id=1177629" class="external">https://bugzilla.opensuse.org/show_bug.cgi?id=1177629</a> to track the kernel traces.</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3363852020-10-15T14:45:59Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>I installed kernel 5.8.14 from kernel:stable repo this morning to see if things improve, or not. So far, it seems a bit better, but not as good as with Leap 15.1.</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3365442020-10-16T11:57:38Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>I reverted qemu to 3.1.1.1 few minutes ago.</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3367512020-10-17T20:43:34Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> changed from <i>2020-10-16</i> to <i>2020-10-24</i></li></ul><p>ok, that would be also my ideas, kernel and qemu. Let's hop it's one of these :)</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3369402020-10-19T06:37:43Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>The nightly update installed again qemu 4.2, so I did it again, plus a lock on <code>qemu-arm</code> package. It is rebooting atm.</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3450852020-10-30T06:03:05Zokurzokurz@suse.com
<ul></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/29593">@ggardet_arm</a> the due date has passed, I would like to call this ticket "Resolved" as we are still running openSUSE Leap 15.2 . There is the lock on "qemu-arm" but other packages are up-to-date. Is there anything more you want to do for this ticket why we should keep it in "In Progress"?</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3454602020-11-02T08:29:56Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>okurz wrote:</p>
<blockquote>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/29593">@ggardet_arm</a> the due date has passed, I would like to call this ticket "Resolved" as we are still running openSUSE Leap 15.2 . There is the lock on "qemu-arm" but other packages are up-to-date. Is there anything more you want to do for this ticket why we should keep it in "In Progress"?</p>
</blockquote>
<p>It is updated but we still have the performance problem...</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3486642020-11-09T13:51:16Zlivdywanliv.dywan@suse.com
<ul><li><strong>Due date</strong> changed from <i>2020-10-24</i> to <i>2020-11-13</i></li></ul> openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3486732020-11-09T13:58:36Zggardet_armguillaume.gardet@arm.com
<ul></ul><p>I added <code>dioread_nolock</code> to the ext4 mount option and it seems to be a bit better, but still not as good as Leap 15.1.</p>
<pre><code>UUID=4816fdde-f472-48fe-b35c-35de23396a50 /var/lib/openqa ext4 noatime,data=writeback,commit=1200,dioread_nolock 0 2
</code></pre> openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3488082020-11-09T21:38:47Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> deleted (<del><i>2020-11-13</i></del>)</li><li><strong>Target version</strong> changed from <i>Ready</i> to <i>future</i></li></ul><p>ok, I see that you did not give up. That's good :) Removing from backlog of SUSE QE Tools team. Feel free to assign back to us whenever you need our help but right now it seems you know better :)</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3571702020-12-09T10:45:08Zokurzokurz@suse.com
<ul><li><strong>Priority</strong> changed from <i>High</i> to <i>Normal</i></li></ul> openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=3573322020-12-09T19:00:21Zokurzokurz@suse.com
<ul><li><strong>Subject</strong> changed from <i>Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"backend died: Migrate to file failed"</i> to <i>Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"</i></li></ul> openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=4041852021-05-05T17:38:12Zokurzokurz@suse.com
<ul></ul><p><a class="user active user-mention" href="https://progress.opensuse.org/users/29593">@ggardet_arm</a> how do you see the current situation in 2021?</p>
openQA Infrastructure - action #72079: Upgrade o3 worker openqa-aarch64 to openSUSE Leap 15.2 (to use newer packages specifically needed for aarch64 and as precursor), also problem auto_review:"(?s)starting: /usr/bin/qemu-system-aarch64.*backend died: Migrate to file failed"https://progress.opensuse.org/issues/72079?journal_id=4042092021-05-05T18:10:32Zggardet_armguillaume.gardet@arm.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul><p>okurz wrote:</p>
<blockquote>
<p><a class="user active user-mention" href="https://progress.opensuse.org/users/29593">@ggardet_arm</a> how do you see the current situation in 2021?</p>
</blockquote>
<p>It works properly now. :)</p>