Project

General

Profile

action #93799

Updated by vpelcak over 2 years ago

## User story 
 There is a new recent feature in template generator. 
 If the automation squad disables the 

 Whenever is openQA testcase in for aggregated openQA tests (for example in order to fix it), unscheduled, this information is automatically detected for shared via the testers template generator. 
 Thing is, that its usability needs to identify missing testcase and cover be improved so that it becomes more useful for the missing tests by manual testing. Update Validation squad. 

 As a tester, I then occasionally encounter stuff like: 

 Example: 

 http://qam.suse.de/testreports/SUSE:Maintenance:19738:241935/log 

 regression tests: 
 ----------------- 
 WARNING: One or more test suites are missing compared to last runs with these source rpms. 
          Please, check openQA manually. Missing test suites are: 

   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-sles4sap_hana_node01 / x86_64 
       sle-12-SP3-Server-DVD-SAP-Incidents-x86_64-Build:19600:libxml2-qam-sles4sap_hana_node01@64bit-sap-qam 
   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-create_hdd_sles4sap_gnome / x86_64 
       sle-12-SP3-Server-DVD-SAP-Incidents-x86_64-Build:19600:libxml2-qam-create_hdd_sles4sap_gnome@64bit 
   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-sles4sap_scc_gnome_hana_cli / x86_64 
       sle-12-SP3-Server-DVD-SAP-Incidents-x86_64-Build:19600:libxml2-qam-sles4sap_scc_gnome_hana_cli@64bit-sap-qam 
   - Maintenance: SLE 12 SP3 Incidents -> qam-incidentinstall / ppc64le 
       sle-12-SP3-Server-DVD-Incidents-Install-ppc64le-Build:19600:libxml2-qam-incidentinstall@ppc64le 
   - Maintenance: SLE 12 SP3 Incidents -> mau-extratests2 / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mau-extratests2@64bit 
   - Maintenance: SLE 12 SP3 Incidents -> qam-incidentinstall / s390x 
       sle-12-SP3-Server-DVD-Incidents-Install-s390x-Build:19600:libxml2-qam-incidentinstall@zkvm 
   - Maintenance: SLE 12 SP3 Incidents -> mau-extratests-zypper / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mau-extratests-zypper@64bit 
   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-incidentinstall-sap / x86_64 
       sle-12-SP3-Server-DVD-SAP-Incidents-Install-x86_64-Build:19600:libxml2-qam-incidentinstall-sap@64bit 
   - Maintenance: SLE 12 SP3 Incidents -> mau-extratests-phub / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mau-extratests-phub@64bit 
   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-sles4sap_scc_gnome_netweaver / x86_64 
       sle-12-SP3-Server-DVD-SAP-Incidents-x86_64-Build:19600:libxml2-qam-sles4sap_scc_gnome_netweaver@64bit 
   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-sles4sap_hana_node02 / x86_64 
       sle-12-SP3-Server-DVD-SAP-Incidents-x86_64-Build:19600:libxml2-qam-sles4sap_hana_node02@64bit-sap-qam 
   - Maintenance: SLE 12 SP3 Incidents -> qam-allpatterns+addons / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-qam-allpatterns+addons@64bit 
   - Maintenance: SLE 12 SP3 Incidents -> mau-webserver / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mau-webserver@64bit 
   - Maintenance: SLE 12 SP3 Incidents -> qam-incidentinstall / x86_64 
       sle-12-SP3-Server-DVD-Incidents-Install-x86_64-Build:19600:libxml2-qam-incidentinstall@64bit 
   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-incidentinstall-sap / ppc64le 
       sle-12-SP3-Server-DVD-SAP-Incidents-Install-ppc64le-Build:19600:libxml2-qam-incidentinstall-sap@ppc64le 
   - Maintenance: SLE 12 SP3 Incidents -> mau-filesystem / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mau-filesystem@64bit 
   - Maintenance: SLE 12 SP3 Incidents -> mau-extratests1 / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mau-extratests1@64bit 
   - Maintenance: SLE 12 SP3 Incidents -> mau-sles-robot-fw / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mau-sles-robot-fw@64bit-2gbram 
   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-sles4sap_scc_gnome_saptune / x86_64 
       sle-12-SP3-Server-DVD-SAP-Incidents-x86_64-Build:19600:libxml2-qam-sles4sap_scc_gnome_saptune@64bit 
   - Maintenance: SLE 12 SP3 Incidents -> mau-extratests-kdump / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mau-extratests-kdump@64bit 
   - Maintenance: SLE 12 SP3 SAP Incidents -> qam-sles4sap_hana_supportserver / x86_64 
       sle-12-SP3-Server-DVD-SAP-Incidents-x86_64-Build:19600:libxml2-qam-sles4sap_hana_supportserver@64bit 
   - Maintenance: SLE 12 SP3 Incidents -> mru-install-minimal-with-addons / x86_64 
       sle-12-SP3-Server-DVD-Incidents-x86_64-Build:19600:libxml2-mru-install-minimal-with-addons@64bit 

 in the testreport. 

 Testers do not understand it and don't know how Tester needs to interpret it. 

 ## Acceptance criteria 
 * **AC1:** it is clear be able to the tester easily figure out what is needed of him in those cases. 
 Does he needs need to do when extra manual testing? What he sees message like the one above 
 * **AC2:** ideally, information about what testing needs to be done to cover for the disabled testcase should be easily accessible 

 I gave this high priority, because there is a danger, that additional testing won't happen and updates are released with reduced test coverage with risk of regressions. test?

Back