QA » openQA Project » openQA Infrastructure

putting back as I'm currently to involved with #136130

SP cannot be reached at the moment. Resetting power to socket 9 on http://epdu-b4.qe.nue2.suse.org/ did not work. Adding next-frankencampus-visit to physically check connections and power cycle the machine if needed.

Will have a look on wednesday, if anyone comes around earlier, please reassign.

Maybe one can try from the grub menu "Enter firmware settings". I tried to add console=tty0 to the linux command line arguments but still nothing shown on the display after "EFI stub: Exiting boot services..."

@dheidler back to you. I suggest try the above with local display and keyboard, experiment with the installed grub menu, try boot over network or from a local USB thumbdrive.

https://openqa.opensuse.org/tests/3680520

This is what I did after installing the OS:

zypper ar -p 95 -f 'http://download.opensuse.org/repositories/devel:openQA/$releasever' devel_openQA
zypper ar -p 90 -f 'http://download.opensuse.org/repositories/devel:openQA:Leap:$releasever/$releasever' devel_openQA_Leap
zypper --gpg-auto-import-keys ref
echo "requires:openQA-worker" > /etc/zypp/systemCheck.d/openqa.check
zypper -n in openQA-worker openQA-auto-update openQA-continuous-update os-autoinst-distri-opensuse-deps swtpm # openQA worker services plus dependencies for openSUSE distri or development repo if added previously
zypper -n in ffmpeg-4  # for using external video encoder as it is already configured on some machines like ow19, ow20 and power8
zypper -n in nfs-client  # For /var/lib/openqa/share
zypper -n in bash-completion vim htop strace systemd-coredump iputils tcpdump bind-utils  # for general tinkering
zypper -n in qemu-uefi-aarch32

# for use of nfs
# echo "openqa1-opensuse:/ /var/lib/openqa/share nfs4 noauto,nofail,retry=30,ro,x-systemd.automount,x-systemd.device-timeout=10m,x-systemd.mount-timeout=30m 0 0" >> /etc/fstab

# for use of cacheservice
# if connecting via ssh-rsync to o3
mkdir -p /var/lib/openqa/ssh/
chown _openqa-worker: /var/lib/openqa/ssh/
sudo -u _openqa-worker ssh-keygen -t ed25519 -N '' -f /var/lib/openqa/ssh/id_ed25519
# append /var/lib/openqa/ssh/id_ed25519.pub to ariel:/var/lib/openqa/.ssh/authorized_keys
sudo -u _openqa-worker ssh -i /var/lib/openqa/ssh/id_ed25519 -p 2214 -o "UserKnownHostsFile /var/lib/openqa/ssh/known_hosts" geekotest@gate.opensuse.org whoami
# note that ariel can be reached directly via ariel.dmz-prg2.suse.org from suse network

sed -i 's/\(solver.dupAllowVendorChange = \)false/\1true/' /etc/zypp/zypp.conf

instances=30 bash -x $(which os-autoinst-setup-multi-machine)

#[global]
#HOST = https://openqa.opensuse.org
#CACHEDIRECTORY = /var/lib/openqa/cache
##WORKER_CLASS = foo
#EXTERNAL_VIDEO_ENCODER_CMD=ffmpeg -y -hide_banner -nostats -r 24 -f image2pipe -vcodec ppm -i - -pix_fmt yuv420p -c:v libvpx-vp9 -crf 35 -b:v 1500k -cpu-used 0
#[https://openqa.opensuse.org]
##TESTPOOLSERVER = -e 'ssh -p 2214 -i /var/lib/openqa/ssh/id_ed25519 -o "UserKnownHostsFile /var/lib/openqa/ssh/known_hosts"' geekotest@gate.opensuse.org:/var/lib/openqa/tests
##TESTPOOLSERVER = -e 'ssh -i /var/lib/openqa/ssh/id_ed25519 -o "UserKnownHostsFile /var/lib/openqa/ssh/known_hosts"' geekotest@ariel.dmz-prg2.suse.org:/var/lib/openqa/tests
#TESTPOOLSERVER = rsync://openqa.opensuse.org/tests

zypper -n in rsync sshpass
rsync -rP --rsh="sshpass -p opensuse ssh -o StrictHostKeyChecking=no" root@openqaworker24:/etc/openqa/ /etc/openqa/
rsync -rP --rsh="sshpass -p opensuse ssh -o StrictHostKeyChecking=no" root@openqaworker24:/etc/wicked/scripts/gre_tunnel_preup.sh /etc/wicked/scripts/
sed -i "s/openqaworker24/$HOSTNAME/g" /etc/openqa/workers.ini
# comment out the worker that we are on and enable all other entries
vi /etc/wicked/scripts/gre_tunnel_preup.sh
# use only hostname as worker class for testing
vi /etc/openqa/workers.ini

chown _openqa-worker /etc/openqa/client.conf
chmod 640 /etc/openqa/client.conf

# configure /etc/openqa/client.conf and /etc/openqa/workers.ini, then enable the desired number of worker slots, e.g.:
systemctl enable --now openqa-worker-auto-restart@{1..30}.service openqa-reload-worker-auto-restart@{1..30}.path openqa-auto-update.timer openqa-continuous-update.timer openqa-worker-cacheservice.service openqa-worker-cacheservice-minion.service

# on aarch64 do something about hugepages permissions (see https://progress.opensuse.org/issues/53234)
echo "hugetlbfs /dev/hugepages hugetlbfs defaults,gid=kvm,mode=1775,pagesize=1G" >> /etc/fstab

# add this to kernel cmdline via /etc/default/grub (& update-bootloader) on aarch64:
# showopts nospec spectre_v2=off pti=off kpti=off mitigations=off default_hugepagesz=1G hugepagesz=1G hugepages=64 crashkernel=167M enforcing=0

Please ensure "AC3: On top of AC1 machine is successfully working on o3 jobs" is covered. https://openqa.opensuse.org/tests/3680520#step/patch_agama/6 looks like tests are executing too slow, maybe not optimized virtualization emulation. If in doubt please ask ggardet in irc://libera.chat/opensuse-factory

dheidler wrote in #note-18:

https://openqa.opensuse.org/tests/3680520

This is what I did after installing the OS:
[…]

Did you actually reinstall the system? If yes, why?

And I see that the system has worker classes "WORKER_CLASS = aarch64-o3,qemu_aarch64,qemu_aarch64_lse", didn't it have 32bit ones in before? See for example https://openqa.opensuse.org/admin/workers/158 , one instance from the same host before the move.

okurz wrote in #note-19:

Please ensure "AC3: On top of AC1 machine is successfully working on o3 jobs" is covered. https://openqa.opensuse.org/tests/3680520#step/patch_agama/6 looks like tests are executing too slow, maybe not optimized virtualization emulation. If in doubt please ask ggardet in irc://libera.chat/opensuse-factory

https://openqa.opensuse.org/tests/3677478#step/agama/3 already had the same error. And that machine is only twice as fast to execute that test. Without kvm we would be much slower than only at 50%.

This machine might have been overloaded as 30 worker slots were enabled.
I reduced the number of slots to 16, like it was the case before on this machine.
Also I did some checks and kvm seems to work. Qemu also doesn't complain about missing /dev/kvm - because it is present.

https://gitlab.suse.de/openqa/monitor-o3/-/merge_requests/12

Please show successful aarch32 jobs for verification. And https://gitlab.suse.de/openqa/monitor-o3/-/pipeline_schedules shows that the last job failed.

Only the overall status. https://gitlab.suse.de/openqa/monitor-o3/-/jobs/1977357 is the only one that doesn't fail due to direct ping.

https://openqa.opensuse.org/tests/3719355
https://openqa.opensuse.org/tests/3719370