Project

General

Profile

Actions

action #120720

open

coordination #68794: [qe-core][functional][epic] rework postfail hooks

[qe-core] Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest

Added by szarate about 2 years ago. Updated 23 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Refactor/Code Improvements
Target version:
-
Start date:
2022-11-18
Due date:
% Done:

0%

Estimated time:
Difficulty:
Sprint:
QE-Core: November Sprint 23 (Nov 15 - Dec 13)

Description

In order for us to shed some time when uploading logs, we can directly cat the logs to the serial console, using script_output and save the contents to the ulogs/ directory in the worker's pool directory, which os-autoinst will upload at the end of the test

This will allow us to save a lot of time and avoid problems with typing when the host is under heavy load.

Tasks

  • Modify save_and_upload_log and tar_and_upload_logs to use save_ulog instead of upload_logs
  • Allow a force_upload flag, so corner cases can be considered

Suggestions

Acceptance Criteria

  1. AC1: os-autoinst uploads the logs properly
  2. AC2: Binary files can be used/viewed properly (images, tar files, executables)
  3. AC3: Large files (500MB) can be uploaded too

Related issues 2 (0 open2 closed)

Related to openQA Tests (public) - action #120723: [qe-core] Guarantee `show_tasks_in_blocked_state` will always be calledResolveddzedro2022-11-18

Actions
Copied to openQA Tests (public) - action #120735: [qe-core] PoC: Decrease typing on `post_fail_hooks` log collectionResolveddvenkatachala2022-11-18

Actions
Actions #1

Updated by szarate about 2 years ago

  • Subject changed from Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest to [qe-core] Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest
Actions #2

Updated by szarate about 2 years ago

  • Copied to action #120735: [qe-core] PoC: Decrease typing on `post_fail_hooks` log collection added
Actions #3

Updated by szarate about 2 years ago

  • Sprint changed from QE-Core: November Sprint (Nov 09 - Dec 07) to QE-Core: December Sprint (Dec 07 - Jan 04)
Actions #4

Updated by dvenkatachala about 2 years ago

  • Assignee set to dvenkatachala
Actions #5

Updated by szarate about 2 years ago

  • Sprint changed from QE-Core: December Sprint (Dec 07 - Jan 11) to QE-Core: January Sprint (Jan 11 - Feb 08)
Actions #6

Updated by dvenkatachala about 2 years ago

  • Status changed from Workable to In Progress
Actions #7

Updated by szarate about 2 years ago

  • Related to action #120723: [qe-core] Guarantee `show_tasks_in_blocked_state` will always be called added
Actions #8

Updated by dvenkatachala about 2 years ago

Modified the functions in lib/Utils/Logging.pm to save the logs in ulogs directory. All files are getting uploaded except the tar files when the test fails. Currently I am working on it.

Actions #9

Updated by szarate almost 2 years ago

  • Sprint changed from QE-Core: January Sprint (Jan 11 - Feb 08) to QE-Core: February Sprint (Feb 08 - Mar 08)
Actions #10

Updated by dvenkatachala almost 2 years ago

I tried to modify the save_and_upload_log function call in the libs/Utils/logging.pm to save logs in the ulogs directory of the worker's pool.
As suggested in the ticket, I modified the function save_and_upload_log to directly cat the logs to the serial console, using script_output and save the contents to the ulogs/ directory in the worker's pool.

This solution works fine for txt file with limited size. With this approach I think there is not much time saving compared with the existing code(verified with single test run)
However in this method cat commands hangs up when there is text/tar file with large size(for e.g., 100MB)

I would suggest to copy the log files directly to worker's pool instead of cat and save the contents.

Actions #11

Updated by dvenkatachala almost 2 years ago

  • Status changed from In Progress to Feedback
Actions #12

Updated by dvenkatachala over 1 year ago

  • Assignee deleted (dvenkatachala)
Actions #13

Updated by szarate about 1 year ago

  • Sprint deleted (QE-Core: February Sprint (Feb 08 - Mar 08))
  • Status changed from Feedback to Workable
Actions #14

Updated by szarate about 1 year ago

  • Sprint set to QE-Core: November Sprint 23 (Nov 15 - Dec 13)
Actions #15

Updated by szarate about 1 year ago

  • Status changed from Workable to New
Actions #16

Updated by szarate about 1 year ago

  • Target version deleted (QE-Core: Ready)
Actions #17

Updated by slo-gin 23 days ago

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Actions

Also available in: Atom PDF