action #120720: [qe-core] Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest - openQA Tests (public) - openSUSE Project Management Tool

Actions

Copy link

action #120720

open

coordination #68794: [qe-core][functional][epic] rework postfail hooks

[qe-core] Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest

Added by szarate about 2 years ago. Updated 11 months ago.

Status:

New

Priority:

Normal

Assignee:

Category:

Refactor/Code Improvements

Target version:

Start date:

2022-11-18

Due date:

% Done:

Estimated time:

Difficulty:

Sprint:

QE-Core: November Sprint 23 (Nov 15 - Dec 13)

Tags:

qe-core-november-sprint

Description

In order for us to shed some time when uploading logs, we can directly cat the logs to the serial console, using script_output and save the contents to the ulogs/ directory in the worker's pool directory, which os-autoinst will upload at the end of the test

This will allow us to save a lot of time and avoid problems with typing when the host is under heavy load.

Tasks¶

Modify save_and_upload_log and tar_and_upload_logs to use save_ulog instead of upload_logs
Allow a force_upload flag, so corner cases can be considered

Suggestions¶

See https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/15264

Acceptance Criteria¶

AC1: os-autoinst uploads the logs properly
AC2: Binary files can be used/viewed properly (images, tar files, executables)
AC3: Large files (500MB) can be uploaded too

Related issues 2 (0 open — 2 closed)

Actions

Copy link

Updated by szarate about 2 years ago

Subject changed from Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest to [qe-core] Use Utils::Logging::save_ulog to redefine upload_logs in opensusebasetest

Actions

Copy link

Updated by szarate about 2 years ago

Copied to action #120735: [qe-core] PoC: Decrease typing on `post_fail_hooks` log collection added

Actions

Copy link

Updated by szarate about 2 years ago

Sprint changed from QE-Core: November Sprint (Nov 09 - Dec 07) to QE-Core: December Sprint (Dec 07 - Jan 04)

Actions

Copy link

Updated by dvenkatachala almost 2 years ago

Assignee set to dvenkatachala

Actions

Copy link

Updated by szarate almost 2 years ago

Sprint changed from QE-Core: December Sprint (Dec 07 - Jan 11) to QE-Core: January Sprint (Jan 11 - Feb 08)

Actions

Copy link

Updated by dvenkatachala almost 2 years ago

Status changed from Workable to In Progress

Actions

Copy link

Updated by szarate almost 2 years ago

Related to action #120723: [qe-core] Guarantee `show_tasks_in_blocked_state` will always be called added

Actions

Copy link

Updated by dvenkatachala almost 2 years ago

Modified the functions in lib/Utils/Logging.pm to save the logs in ulogs directory. All files are getting uploaded except the tar files when the test fails. Currently I am working on it.

Actions

Copy link

Updated by szarate almost 2 years ago

Sprint changed from QE-Core: January Sprint (Jan 11 - Feb 08) to QE-Core: February Sprint (Feb 08 - Mar 08)

Actions

Copy link

#10

Updated by dvenkatachala almost 2 years ago

I tried to modify the save_and_upload_log function call in the libs/Utils/logging.pm to save logs in the ulogs directory of the worker's pool.
As suggested in the ticket, I modified the function save_and_upload_log to directly cat the logs to the serial console, using script_output and save the contents to the ulogs/ directory in the worker's pool.

This solution works fine for txt file with limited size. With this approach I think there is not much time saving compared with the existing code(verified with single test run)
However in this method cat commands hangs up when there is text/tar file with large size(for e.g., 100MB)

I would suggest to copy the log files directly to worker's pool instead of cat and save the contents.

Actions

Copy link

#11