Project

General

Profile

Actions

action #49115

closed

[functional][u][userspace] Convert "qa_userspace_php7" to "console/php7"

Added by okurz about 5 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
Enhancement to existing tests
Target version:
SUSE QA - Milestone 30
Start date:
2019-03-12
Due date:
% Done:

0%

Estimated time:
42.00 h
Difficulty:

Description

Motivation

In #44138 we changed most userspace scenarios to use QA_TESTSET to provide better to understand test results on failures. The scenarios that so far could not be converted should be handled the same

We agreed to not maintain qa_userspace tests. To not loose coverage, we wanted to migrate them to normal openQA test modules.

There are already three existing modules that test php7:

  • console/php7
  • console/php7_mysql
  • console/php7_postgresql

Acceptance criteria

  • AC1: testsuite qa_userspace_php7 is covered by unit tests in OBS package, or openQA modules console/php7, console/mysql or console/php7_postgresql

Tasks

  1. Investigate what is covered by qa_userspace_php7.
  2. Implement what is possible at OBS package level
  3. Implement the rest as normal openQA module.

Related issues 2 (0 open2 closed)

Blocks openQA Tests - action #44882: [functional][sle][userspace][u]test fails in execute_test_run - almost all php tests failedRejectedmgriessmeier2018-12-07

Actions
Copied from openQA Tests - action #49112: [functional][u][userspace] Convert "qa_userspace_sharutils" to "console/sharutils"Resolveddheidler2019-03-12

Actions
Actions #1

Updated by okurz about 5 years ago

  • Copied from action #49112: [functional][u][userspace] Convert "qa_userspace_sharutils" to "console/sharutils" added
Actions #2

Updated by okurz about 5 years ago

  • Blocks action #44882: [functional][sle][userspace][u]test fails in execute_test_run - almost all php tests failed added
Actions #3

Updated by mgriessmeier over 4 years ago

  • Target version changed from Milestone 26 to Milestone 30+
Actions #4

Updated by SLindoMansilla over 4 years ago

  • Priority changed from Low to High
Actions #5

Updated by SLindoMansilla over 4 years ago

  • Subject changed from [functional][u][userspace] Convert "php/php7" to QA_TESTSUITE to [functional][u][userspace] Convert "qa_userspace_php7" to "console/php7"
  • Description updated (diff)
Actions #6

Updated by SLindoMansilla over 4 years ago

  • Description updated (diff)
Actions #7

Updated by SLindoMansilla over 4 years ago

  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from High to Low
  • Target version changed from Milestone 30+ to Milestone 28
  • Estimated time set to 42.00 h
Actions #8

Updated by dheidler over 4 years ago

  • Status changed from Workable to Resolved
  • Assignee set to dheidler

This testsuite doesn't seem to exist anymore.

[dheidler@artemis openqa-tests]$ find | grep qa_userspace_php
[dheidler@artemis openqa-tests]$ git grep qa_userspace
[dheidler@artemis openqa-tests]$ 
Actions #9

Updated by okurz over 4 years ago

  • Status changed from Resolved to Workable

you are calling it "testsuite" correctly but you look for test modules. Try https://openqa.suse.de/admin/test_suites and please do not simply accept a "oh, I could not find the test" as "Resolved". You are paid to ensure that SLE has a good quality. And if for example we lost test coverage unknowingly this should not be accepted with simply a shrug.

Actions #10

Updated by dheidler over 4 years ago

  • Status changed from Workable to In Progress

Ah sorry - I thought this was refering to a module.

Actions #11

Updated by okurz over 4 years ago

I understand the confusion. It comes from the rename of the subject that slindomansilla did. It's apparently confusing and also a bit to too prescriptive of the implementation. Also obviously one can not convert a testsuite "qa_userspace_php7" to a test module "console/php7", which by the way already exists and is also mentioned in the description. The decision to "not maintain qa_userspace tests" in favor of plain os-autoinst tests is questionable but the important part is valid: "To not loose coverage" :)

Actions #12

Updated by dheidler over 4 years ago

I'm not able to find a recent job (or even a job at all) that is executing the qa_userspace_php7 testsuite.
So we don't seem to even test that one currently.
I am able to find similar tests that test eg. gzip.
That gzip test uses QA_TESTSET insted of the QA_TESTSUITE var though.

Actions #13

Updated by dheidler over 4 years ago

This testsuite contains 2900 regression test cases for upstream bugs.
Some of them are over 10 years old.

Actions #14

Updated by dheidler over 4 years ago

I think we should stop using that testsuite.
PHP has an upstream testsuite: https://travis-ci.org/php/php-src
And this testsuite was not touched for more than two years: https://github.com/SUSE/qa-testsuites/tree/master/tests/qa_test_php

At the same time I will extend the existing tests/console/php7.pm module to do a bit more.

Actions #15

Updated by dheidler over 4 years ago

  • Status changed from In Progress to Feedback
Actions #16

Updated by mgriessmeier about 4 years ago

  • Target version changed from Milestone 28 to Milestone 30

waiting for PR review from zluo
discuss next week

Actions #17

Updated by zluo about 4 years ago

PR merged now.

Actions #18

Updated by dheidler about 4 years ago

  • Status changed from Feedback to Resolved
Actions

Also available in: Atom PDF