Project

General

Profile

action #120720

coordination #68794: [qe-core][functional][epic] rework postfail hooks

[qe-core] Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest

Added by szarate 4 months ago. Updated about 2 months ago.

Status:
Feedback
Priority:
Normal
Assignee:
dvenkatachala
Category:
Refactor/Code Improvements
Target version:
QA - QE-Core: Ready
Start date:
2022-11-18
Due date:
% Done:

0%

Estimated time:
Difficulty:
Tags:
qe-core-november-sprint

Description

In order for us to shed some time when uploading logs, we can directly cat the logs to the serial console, using script_output and save the contents to the ulogs/ directory in the worker's pool directory, which os-autoinst will upload at the end of the test

This will allow us to save a lot of time and avoid problems with typing when the host is under heavy load.

Tasks

  • Modify save_and_upload_log and tar_and_upload_logs to use save_ulog instead of upload_logs
  • Allow a force_upload flag, so corner cases can be considered

Suggestions

Acceptance Criteria

  1. AC1: os-autoinst uploads the logs properly
  2. AC2: Binary files can be used/viewed properly (images, tar files, executables)
  3. AC3: Large files (500MB) can be uploaded too

Related issues

Related to openQA Tests - action #120723: [qe-core] Guarantee `show_tasks_in_blocked_state` will always be calledResolved2022-11-18

Copied to openQA Tests - action #120735: [qe-core] PoC: Decrease typing on `post_fail_hooks` log collectionIn Progress2022-11-18

History

#1 Updated by szarate 4 months ago

  • Subject changed from Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest to [qe-core] Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest

#2 Updated by szarate 4 months ago

  • Copied to action #120735: [qe-core] PoC: Decrease typing on `post_fail_hooks` log collection added

#3 Updated by szarate 4 months ago

  • Sprint changed from QE-Core: November Sprint (Nov 09 - Dec 07) to QE-Core: December Sprint (Dec 07 - Jan 04)

#4 Updated by dvenkatachala 3 months ago

  • Assignee set to dvenkatachala

#5 Updated by szarate 3 months ago

  • Sprint changed from QE-Core: December Sprint (Dec 07 - Jan 11) to QE-Core: January Sprint (Jan 11 - Feb 08)

#6 Updated by dvenkatachala 2 months ago

  • Status changed from Workable to In Progress

#7 Updated by szarate 2 months ago

  • Related to action #120723: [qe-core] Guarantee `show_tasks_in_blocked_state` will always be called added

#8 Updated by dvenkatachala about 2 months ago

Modified the functions in lib/Utils/Logging.pm to save the logs in ulogs directory. All files are getting uploaded except the tar files when the test fails. Currently I am working on it.

#9 Updated by szarate about 2 months ago

  • Sprint changed from QE-Core: January Sprint (Jan 11 - Feb 08) to QE-Core: February Sprint (Feb 08 - Mar 08)

#10 Updated by dvenkatachala about 2 months ago

I tried to modify the save_and_upload_log function call in the libs/Utils/logging.pm to save logs in the ulogs directory of the worker's pool.
As suggested in the ticket, I modified the function save_and_upload_log to directly cat the logs to the serial console, using script_output and save the contents to the ulogs/ directory in the worker's pool.

This solution works fine for txt file with limited size. With this approach I think there is not much time saving compared with the existing code(verified with single test run)
However in this method cat commands hangs up when there is text/tar file with large size(for e.g., 100MB)

I would suggest to copy the log files directly to worker's pool instead of cat and save the contents.

#11 Updated by dvenkatachala about 2 months ago

  • Status changed from In Progress to Feedback

Also available in: Atom PDF