action #71602
open[qe-core][functional][osd] default "gnome" test suite on osd has no video due to max_job_time above default which is only necessary in some cases
100%
Description
Observation¶
<fcrozat> are video still recorded for staging tests ? I can't find the link in https://openqa.suse.de/tests/4710118#downloads
<okurz> https://openqa.suse.de/tests/4710118/file/worker-log.txt shows that there was NOVIDEO=1 so no video recorded
<dzedro> Frédéric Crozat there have been change, you have to set NOVIDEO=0 to have video
<okurz> that should not be necessary by default, only if MAX_JOB_TIME exceeds the default of 2h
<okurz> so someone set a MAX_JOB_TIME>2h on the generic "gnome" test suite which then in turn disables the video by default. I recommend to delete the MAX_JOB_TIME again. Normally there should really be no need to exceed 2h
<dzedro> I guess it's bcause of aarch64 https://openqa.suse.de/tests/4702106 or some oher arch, as aarch64 has MAX_JOB_TIME=10800
<okurz> hm, if it's two times slower on aarch64 should not mean we need to have the higher timeout on the test suite. I guess setting MAX_JOB_TIME=10800 on the aarch64 machines is a better approach. I found out in the audit database that it was Dominik Heidler changing that 2019-12-02 . Dominik Heidler WDYT could you set the MAX_JOB_TIME on the slow machines rather than test suites?
Acceptance criteria¶
- AC1: Default test suites, at least "gnome", does not need an explicit MAX_JOB_TIME specified anymore on the test suite
- AC2: MAX_JOB_TIME is only specified where needed
- AC3: Staging tests coverage is unchanged, e.g. the same test modules are still scheduled as part of staging tests independant of the actual scenario
Tasks¶
- Move test modules from gnome that are not directly related to gnome, like mysql, sssd, etc. to an according "extra_tests" test suite
Updated by dheidler over 3 years ago
Wouldn't that disable video completely by default on aarch64?
Updated by dheidler over 3 years ago
I guess it could make more sense to set NOVIDEO=0 in the gnome testsuite.
Updated by okurz over 3 years ago
Probably the best solution would be to ensure that the default "gnome" testsuite is well below 2h even with post_fail_hooks running by moving test modules to specific extra tests. Then no MAX_JOB_TIME is needed at all and no problem with the video exclusion happens.
Updated by szarate over 3 years ago
- Related to coordination #68794: [qe-core][functional][epic] rework postfail hooks added
Updated by ggardet_arm over 3 years ago
- Related to action #71716: No video uploaded when `aarch64_cpu_max` is used added
Updated by SLindoMansilla over 3 years ago
- Description updated (diff)
- Status changed from New to Workable
- Assignee deleted (
SLindoMansilla) - Target version set to Milestone 30
- Estimated time set to 42.00 h
Updated by dheidler over 3 years ago
- Status changed from Workable to In Progress
- Assignee set to dheidler
Updated by dheidler over 3 years ago
- Status changed from In Progress to Blocked
Blocked by #76870 - deprecated non-YAML scheduled extra_tests_on_… testsuites should not be touched anymore.
Updated by dheidler over 3 years ago
- Blocked by action #76870: O3: Replace extra_tests_on_gnome with YAML scheduled extra_tests_gnome added
Updated by tjyrinki_suse over 3 years ago
- Subject changed from [functional][u][osd] default "gnome" test suite on osd has no video due to max_job_time above default which is only necessary in some cases to [qe-core][functional][osd] default "gnome" test suite on osd has no video due to max_job_time above default which is only necessary in some cases
Updated by dheidler over 3 years ago
- Status changed from Blocked to In Progress
The gnome testsuite seems to have already about the same runtime as the extra_tests_gnome one.
So we would only shift the problem by moving test modules.
Therefore I will go with the NOVIDEO=0 approach.
Updated by dheidler over 3 years ago
- Status changed from In Progress to Feedback
Updated by okurz over 3 years ago
dheidler wrote:
The
gnometestsuite seems to have already about the same runtime as theextra_tests_gnomeone.
So we would only shift the problem by moving test modules.
Therefore I will go with theNOVIDEO=0approach.
That does not make sense to me. The extra_tests scenarios have the benefit of being more independant. And gnome is one of the scenarios also used in staging. This is what this ticket is about. Can you please see the specific suggestion "Remove test modules from gnome that are not directly related to gnome, like mysql, sssd, etc."? Of course they don't need to go into extra_tests_gnome, they could also go into something like extra_tests_services or so.
Updated by dheidler over 3 years ago
But wouldn't that mean that we would loose test coverage in staging?
Updated by okurz over 3 years ago
dheidler wrote:
But wouldn't that mean that we would loose test coverage in staging?
true. This means that staging tests likely want a setup for "extra tests" as well or handle it differently in the scenarios for staging and non-staging
Updated by dheidler about 3 years ago
- Status changed from Feedback to Workable
- Assignee deleted (
dheidler)
Updated by tjyrinki_suse almost 3 years ago
- Priority changed from High to Normal
- Target version deleted (
Milestone 30) - Start date deleted (
2020-09-21)
Updated by apappas about 2 years ago
- Status changed from Workable to Feedback
This ticket was discussed in the qe-core refinement session.
The observation and criteria are unclear and the ticket is stale. To move on this we need info:
- Is the problem behind the ticket still happening?
- Could you please format the observance and acceptance criteria better and more verbosely?
Updated by okurz about 2 years ago
- Description updated (diff)
apappas wrote:
This ticket was discussed in the qe-core refinement session.
The observation and criteria are unclear and the ticket is stale. To move on this we need info:
- Is the problem behind the ticket still happening?
Yes, the gnome test suite still has MAX_JOB_TIME=10800 explicitly specified. The only thing that changed is that as noted in #71602#note-12 dheidler has added NOVIDEO=0 which I have responded to in #71602#note-14 stating that this was not the point.
Following https://openqa.suse.de/tests?match=gnome one can find recent examples. For example sle-15-SP4-Online-E-Staging-x86_64-BuildE.83.1-gnome@64bit is a recent job that has MAX_JOB_TIME of 10800 although it (only) took 57 minutes. The job does have a video due to the NOVIDEO=0 setting but this is not a good long-term maintainable solution.
- Could you please format the observance and acceptance criteria better and more verbosely?
Sure, I reformated the ticket description. The chat quote which I put into a "code" section was missing the closing markers. Fixed that. Added acceptance criteria and extended the suggestion.