Project

General

Profile

action #19362

[userspace] split qa_run.pm

Added by yosun over 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Enhancement to existing tests
Target version:
-
Start date:
2018-01-12
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Now if the qa_run.pm related testcases fails not in what they tested, it'll hard to judge where it fails with. To split qa_run.pm into process is a solution for that.

Split it in to:

  • Booting the OS
  • Preparation test package
  • Run test process
  • Upload log and analysis process It's a basic idea, the real change will depends. But the target is fixed to solve the debug problem.

Subtasks

action #30207: [userspace][kernel]Need to install more packages in install_test_suite.pmClosedyosun


Related issues

Related to openQA Tests - action #17872: [sles][functional] sles12_qa_acceptance_fs_stress times out after 2h with no useful informationResolved2017-03-22

History

#1 Updated by jtzhao over 4 years ago

I think you can just read the autoinst-log.txt to know which part failed.

#2 Updated by yosun over 4 years ago

From this related issue you'll know what I mean: https://progress.opensuse.org/issues/17872

It really don't have any useful information in log, if it fails in boot part. What we need to do is to let reviewer more easy to tell if it's a fail in our testsuite. If not, the issue will keep bother us a lot in the future. Oliver and Richard also complained this in last week.
https://openqa.suse.de/tests/829634

#3 Updated by okurz over 4 years ago

  • Related to action #17872: [sles][functional] sles12_qa_acceptance_fs_stress times out after 2h with no useful information added

#4 Updated by okurz over 4 years ago

  • Category set to Enhancement to existing tests

#5 Updated by okurz almost 4 years ago

  • Blocks action #18936: [tools][sles][functional] Enable 3 stress acceptance on s390x added

#6 Updated by okurz almost 4 years ago

yosun any progress here? riafarov has an idea about a PR that he could provide.

#7 Updated by okurz almost 4 years ago

  • Subject changed from [sles][functional] split qa_run.pm to [sle][functional] split qa_run.pm
  • Due date set to 2017-10-25

to unblock #18936 as requested by cachen. riafarov and me want to help and can provide a PR soon I expect.

#9 Updated by yosun almost 4 years ago

  • Subject changed from [sle][functional] split qa_run.pm to [sle][functional][userspace] split qa_run.pm

Add [userspace] tag to trace.

#10 Updated by yosun almost 4 years ago

riafarov, reviewed, and copy some explain of this task here.

Actually the task of split qa_run is for review result more easy, and it's more than just move booting steps out. But you picked the best way to modify it in this PR. Fails before booting success is not failed by what it test, it fails because of OS install or system booting. --> what solved by this PR https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3695

And fails from booting success to testsuite install finish, is another period not failed by what it test, it fails because of network issue or module issue --> this part I'll do it inside of qa_run.pm

Then fails from testsuite install finish to testsuite start running is because of testsuite internal issue. It means qa_test_* need to be fixed. --> also for me todo in split qa_run.pm

And fails from testsuite run to run finish is what we expected product fail/bug.

#11 Updated by riafarov almost 4 years ago

  • Blocks deleted (action #18936: [tools][sles][functional] Enable 3 stress acceptance on s390x)

#12 Updated by riafarov almost 4 years ago

  • Due date deleted (2017-10-25)

yosun thanks for the feedback. Please, proceed with parts which are still missing. And we will proceed with enabling stress tests on s390x.

#13 Updated by sebchlad almost 4 years ago

  • Subject changed from [sle][functional][userspace] split qa_run.pm to [userspace] split qa_run.pm

It seem to me like there is noting else to do on QA SLE Functional, right?

If I'm wrong please add the ticket to the SLE Functional backlog.

#14 Updated by okurz almost 4 years ago

yes, we did "our part" so I am fine with removing the tag again although in future we should rather create specific subtickets and resolve them then and not add/remove tags.

#15 Updated by sebchlad over 3 years ago

  • Assignee changed from yosun to okurz

@Oli: can I ask you to take care of this going forward please?

Reason: as we discussed Sero will focus fulls on kernel&network and you and the QA SLE Func. could see if you could take care of userspace domain.

#16 Updated by yosun over 3 years ago

#17 Updated by okurz over 3 years ago

  • Status changed from New to Resolved
  • Assignee changed from okurz to cachen

Yes, agreed.

Also available in: Atom PDF