action #25216


Transient errors and poor perf for PowerPC workers used by Snapshot

Added by michel_mno almost 7 years ago. Updated almost 7 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:


As per irc chat with DimStar extracted below,
there are problems on Power8 machines used for PowerPC workers used by OBS.

Could we have an analysing of the Power8 machines configurations (memory, perf, OS release/version)
that could explain those problems ?

10:32:40) DimStar: Good morning; I wanted to let you know that I'm going to remove ppc64le from the staging process - there are way too many random/weird fails and the build performance is 'low' - generally it slows down TW stagings, which in turn has negative impact on SLE
(10:42:25) michel_mno: Hello, I assume this will create more build errors detected only in  the Factory::PowerPC
(10:47:28) DimStar: Sadly, yes - this is to be expected. But the fact that it's constantly the slowest arch to build, and the only one requesting me to trigger a build 10 times to get a successful build, makes it really painpuf
(10:47:45) DimStar: I will keep it in openSUSE:Factory:Rings - so it's also visible there
(11:01:07) DimStar: and if the situation improves, we can consider taking it in again (meaning: the rings do not show random broken build fails - and build performance is sort-of usable)
(11:02:21) michel_mno: DimStar who has access to those builders and responsible for their setup ?
(11:02:35) DimStar: on OBS? Adrian and his team
(11:02:40) michel_mno: I did not knew their were performance problem on them
(11:03:37) DimStar: would be greay if it turned out 'just' to be a config problem there
(11:04:20) DimStar: is one of the 'most annoying' packages failing on ppc64le (it takes ~ 30 minutes, and then fails 'a lot')
(11:09:32) michel_mno: I will check above; FYI  yesterday I issued an  osc build  of Staging:A hdf5 and it passed without retry  (it last 3994s)
(11:09:59) DimStar: lucky you :)
(11:11:06) DimStar: :J:DVD/hdf5 -> so far triggered 16 times - I would br REALLY surprised if anything in :J impacts the capability of hdf5 to build (it's a KDE stack upggrade)
(11:11:35) DimStar: K:DVD/hdf5 so far triggered 4 times
Actions #1

Updated by michel_mno almost 7 years ago

Trying to isolate the problem without access to related Power8 Host is not possible !
I need more information on the machine setup.

  • What is installed OS/version ?
  • What are ~/.oscrc build parameters ?
  • Could I have a tarball of /var/log/systemd-journal and dmesg ?

Also available in: Atom PDF