action #49115

[functional][u][userspace] Convert "qa_userspace_php7" to "console/php7"

Added by okurz about 1 year ago. Updated 3 months ago.

Status:ResolvedStart date:12/03/2019
Priority:LowDue date:
Assignee:dheidler% Done:

0%

Category:Enhancement to existing testsEstimated time:42.00 hours
Target version:SUSE QA tests - Milestone 30
Difficulty:
Duration:

Description

Motivation

In #44138 we changed most userspace scenarios to use QA_TESTSET to provide better to understand test results on failures. The scenarios that so far could not be converted should be handled the same

We agreed to not maintain qa_userspace tests. To not loose coverage, we wanted to migrate them to normal openQA test modules.

There are already three existing modules that test php7:

  • console/php7
  • console/php7_mysql
  • console/php7_postgresql

Acceptance criteria

  • AC1: testsuite qa_userspace_php7 is covered by unit tests in OBS package, or openQA modules console/php7, console/mysql or console/php7_postgresql

Tasks

  1. Investigate what is covered by qa_userspace_php7.
  2. Implement what is possible at OBS package level
  3. Implement the rest as normal openQA module.

Related issues

Blocks openQA Tests - action #44882: [functional][sle][userspace][u]test fails in execute_test... Blocked 07/12/2018
Copied from openQA Tests - action #49112: [functional][u][userspace] Convert "qa_userspace_sharutil... Workable 12/03/2019

History

#1 Updated by okurz about 1 year ago

  • Copied from action #49112: [functional][u][userspace] Convert "qa_userspace_sharutils" to "console/sharutils" added

#2 Updated by okurz about 1 year ago

  • Blocks action #44882: [functional][sle][userspace][u]test fails in execute_test_run - almost all php tests failed added

#3 Updated by mgriessmeier 8 months ago

  • Target version changed from Milestone 26 to Milestone 30+

#4 Updated by SLindoMansilla 6 months ago

  • Priority changed from Low to High

#5 Updated by SLindoMansilla 6 months ago

  • Subject changed from [functional][u][userspace] Convert "php/php7" to QA_TESTSUITE to [functional][u][userspace] Convert "qa_userspace_php7" to "console/php7"
  • Description updated (diff)

#6 Updated by SLindoMansilla 6 months ago

  • Description updated (diff)

#7 Updated by SLindoMansilla 6 months ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from High to Low
  • Target version changed from Milestone 30+ to Milestone 28
  • Estimated time set to 42.00

#8 Updated by dheidler 4 months ago

  • Status changed from Workable to Resolved
  • Assignee set to dheidler

This testsuite doesn't seem to exist anymore.

[dheidler@artemis openqa-tests]$ find | grep qa_userspace_php
[dheidler@artemis openqa-tests]$ git grep qa_userspace
[dheidler@artemis openqa-tests]$ 

#9 Updated by okurz 4 months ago

  • Status changed from Resolved to Workable

you are calling it "testsuite" correctly but you look for test modules. Try https://openqa.suse.de/admin/test_suites and please do not simply accept a "oh, I could not find the test" as "Resolved". You are paid to ensure that SLE has a good quality. And if for example we lost test coverage unknowingly this should not be accepted with simply a shrug.

#10 Updated by dheidler 4 months ago

  • Status changed from Workable to In Progress

Ah sorry - I thought this was refering to a module.

#11 Updated by okurz 4 months ago

I understand the confusion. It comes from the rename of the subject that slindomansilla did. It's apparently confusing and also a bit to too prescriptive of the implementation. Also obviously one can not convert a testsuite "qa_userspace_php7" to a test module "console/php7", which by the way already exists and is also mentioned in the description. The decision to "not maintain qa_userspace tests" in favor of plain os-autoinst tests is questionable but the important part is valid: "To not loose coverage" :)

#12 Updated by dheidler 4 months ago

I'm not able to find a recent job (or even a job at all) that is executing the qa_userspace_php7 testsuite.
So we don't seem to even test that one currently.
I am able to find similar tests that test eg. gzip.
That gzip test uses QA_TESTSET insted of the QA_TESTSUITE var though.

#13 Updated by dheidler 4 months ago

This testsuite contains 2900 regression test cases for upstream bugs.
Some of them are over 10 years old.

#14 Updated by dheidler 4 months ago

I think we should stop using that testsuite.
PHP has an upstream testsuite: https://travis-ci.org/php/php-src
And this testsuite was not touched for more than two years: https://github.com/SUSE/qa-testsuites/tree/master/tests/qa_test_php

At the same time I will extend the existing tests/console/php7.pm module to do a bit more.

#15 Updated by dheidler 4 months ago

  • Status changed from In Progress to Feedback

#16 Updated by mgriessmeier 3 months ago

  • Target version changed from Milestone 28 to Milestone 30

waiting for PR review from zluo
discuss next week

#17 Updated by zluo 3 months ago

PR merged now.

#18 Updated by dheidler 3 months ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF