Actions
action #178183
closedcoordination #154768: [saga][epic][ux] State-of-art user experience for openQA
coordination #154771: [epic] Improved test developer user experience
[sporadic] Failing OBS package check t/01_run.t for perl-Mojo-IOLoop-ReadWriteProcess on aarch64+ppc64le size:S
Start date:
2025-03-03
Due date:
2025-03-28
% Done:
0%
Estimated time:
Description
Observation¶
[ 52s] TEST error print
[ 52s]
[ 52s] # Failed test 'process is shut down via kill_whole_group'
[ 52s] # at t/01_run.t line 500.
[ 52s] # got: '1'
[ 52s] # expected: '0'
[ 52s] # Looks like you failed 1 test of 1.
[ 52s]
[ 52s] # Failed test 'stop_whole_process_group_gracefully'
[ 52s] # at t/01_run.t line 501.
[ 52s] TEST error print
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork(): Fork: {
[ 52s] use warnings;
[ 52s] use strict;
[ 52s] shift();
[ 52s] print "$_$/" foreach (@_);
[ 52s] }
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Exit status: 248
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Returns: nothing
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Errors:
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_shutdown(): Shutdown 1540
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork(): Fork: {
[ 52s] use warnings;
[ 52s] use strict;
[ 52s] shift();
[ 52s] print "$_$/" foreach (@_);
[ 52s] }
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Exit status: 144
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Returns: nothing
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Errors:
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_shutdown(): Shutdown 1541
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork(): Fork: {
[ 52s] use warnings;
[ 52s] use strict;
[ 52s] shift();
[ 52s] print "$_$/" foreach (@_);
[ 52s] }
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Exit status: 152
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Returns: nothing
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Errors:
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_shutdown(): Shutdown 1542
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork(): Fork: {
[ 52s] use warnings;
[ 52s] use strict;
[ 52s] shift();
[ 52s] print "$_$/" foreach (@_);
[ 52s] }
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Exit status: 112
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Returns: nothing
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Errors:
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_shutdown(): Shutdown 1543
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork(): Fork: {
[ 52s] use warnings;
[ 52s] use strict;
[ 52s] is process('execute', '/bin/true')->quirkiness(1)->start->wait_stop->exit_status, 0, 'process(execute) from process(code) -- retval check true';
[ 52s] is process('execute', '/bin/false')->quirkiness(1)->start->wait_stop->exit_status, 1, 'process(execute) from process(code) -- retval check false';
[ 52s] is process(sub {
[ 52s] print 'sub-sub-process';
[ 52s] }
[ 52s] )->start->wait_stop->read_all_stdout, 'sub-sub-process', 'process(code) works from process(code)';
[ 52s] print 'DONE';
[ 52s] }
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Exit status: 192
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Returns: 1
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Errors:
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_shutdown(): Shutdown 1544
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_open(): Execute: '/I/do/not/exist'
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Exit status: 255
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_open(): Execute: 'sleep 0.2'
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_open_collect_status(): Forked code Process Exit status: 0
[ 52s] >> Mojo::IOLoop::ReadWriteProcess::_open(): Execute: 'perl', '/home/abuild/rpmbuild/BUILD/perl-Mojo-IOLoop-ReadWriteProcess-0.340.0-build/Mojo-IOLoop-ReadWriteProcess-0.34/t/data/simple_rwp.pl'
[ 53s] >> Mojo::IOLoop::ReadWriteProcess::_open_collect_status(): Forked code Process Exit status: 0
[ 53s] >> Mojo::IOLoop::ReadWriteProcess::_shutdown(): Shutdown 1550
[ 53s] >> Mojo::IOLoop::ReadWriteProcess::_open(): Execute: '/home/abuild/rpmbuild/BUILD/perl-Mojo-IOLoop-ReadWriteProcess-0.340.0-build/Mojo-IOLoop-ReadWriteProcess-0.34/t/data/sigchld_handler.pl'
[ 53s] >> Mojo::IOLoop::ReadWriteProcess::_open_collect_status(): Forked code Process Exit status: 0
[ 53s] >> Mojo::IOLoop::ReadWriteProcess::_shutdown(): Shutdown 1552
[ 53s] # Looks like you failed 1 test of 12.
[ 53s] t/01_run.t ...............
Suggestions¶
- Mitigate by removing the check based on architecture
- Consider reporting an upstream issue in https://github.com/openSUSE/Mojo-IOLoop-ReadWriteProcess/ but be aware that it is mostly us anyway that would need to fix such issues
- Consider workarounds within the test module
- Be aware about #176148
- Fix the root cause
- Remove mitigations
- Ensure stability within multi-arch OBS checks
- Confirm if this also affects GitHub CI e.g. by proposing a trivial PR and checking the result
Updated by okurz about 2 months ago
- Copied to action #178186: [sporadic] Failing OBS package check t/12_mocked_container.t for perl-Mojo-IOLoop-ReadWriteProcess on aarch64 size:S added
Updated by tinita about 1 month ago
- Subject changed from [sporadic] Failing OBS package check t/01_run.t for perl-Mojo-IOLoop-ReadWriteProcess on aarch64+ppc64le to [sporadic] Failing OBS package check t/01_run.t for perl-Mojo-IOLoop-ReadWriteProcess on aarch64+ppc64le size:S
- Description updated (diff)
- Status changed from New to Workable
Updated by mkittler about 1 month ago
- Status changed from Workable to In Progress
- Assignee set to mkittler
Updated by openqa_review about 1 month ago
- Due date set to 2025-03-28
Setting due date based on mean cycle time of SUSE QE Tools
Updated by mkittler about 1 month ago
- Status changed from In Progress to Feedback
This PR might help to fix this: https://github.com/openSUSE/Mojo-IOLoop-ReadWriteProcess/pull/66
It has already been merged and is present on https://build.opensuse.org/package/show/devel:openQA/perl-Mojo-IOLoop-ReadWriteProcess.
Let's see how the package build behaves after that. I have already tested it on my home project (https://build.opensuse.org/package/show/home:mkittler:branches:devel:languages:perl/perl-Mojo-IOLoop-ReadWriteProcess).
Updated by mkittler about 1 month ago ยท Edited
Looks like it still fails sometimes:
[ 26s] # Failed test 'process is shut down via kill_whole_group'
[ 26s] # at t/01_run.t line 510.
[ 26s] # got: '1'
[ 26s] # expected: '0'
[ 26s] # Looks like you failed 1 test of 1.
On my home project now even both archs fail again (after I initially wasn't able to reproduce the problem on aarch64 anymore):
[ 22s] # waiting until process group has been created
[ 22s] # test pid: 1443, gpid: 1171
[ 22s] # sub process pid: 1482, gpid: 1482
[ 22s] # waiting until all sub processes have been forked
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::stop(): Stopping -1482
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::stop(): attempt 1/1 to kill process: -1482, timeout: 0.05
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::send_signal(): Sending signal '15' to -1482
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::stop(): Could not kill process id: -1482, blocking attempt
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::send_signal(): Sending signal '9' to -1482
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::_shutdown(): Shutdown 1482
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Exit status: 0
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Returns: nothing
[ 22s] >> Mojo::IOLoop::ReadWriteProcess::_fork_collect_status(): Forked code Process Errors:
[ 22s] not ok 1 - process is shut down via kill_whole_group
[ 22s]
[ 22s] # Failed test 'process is shut down via kill_whole_group'
[ 22s] # at t/01_run.t line 511.
[ 22s] # got: '1'
[ 22s] # expected: '0'
[ 22s] 1..1
[ 22s] # Looks like you failed 1 test of 1.
[ 22s] not ok 7 - stop_whole_process_group_gracefully
[ 22s]
[ 22s] # Failed test 'stop_whole_process_group_gracefully'
[ 22s] # at t/01_run.t line 512.
Updated by mkittler about 1 month ago
Looks like it can also fail on a different place:
[ 420s] # Failed test 'Process now is not running'
[ 420s] # at t/01_run.t line 110.
[ 420s] # got: '1'
[ 420s] # expected: '0'
[ 421s] # Looks like you failed 1 test of 7.
[ 421s]
[ 421s] # Failed test 'process is_running()'
[ 421s] # at t/01_run.t line 129.
[ 427s] TEST error print
(This is from an s390x build.)
Updated by mkittler about 1 month ago
Updated by mkittler about 1 month ago
- Status changed from Workable to In Progress
Updated by mkittler about 1 month ago
- Status changed from In Progress to Resolved
I went through all occurrences where we check whether the process is running or not running anymore in t/01_run.t. So this kind of problem should be fixed now.
The build looks now good on https://build.opensuse.org/package/show/devel:openQA/perl-Mojo-IOLoop-ReadWriteProcess. With that I'm resolving the ticket. If there's still something missing after all we can re-open it (or create a new ticket if it is a different kind of failure).
Actions