Project

General

Profile

action #121579

Updated by okurz almost 2 years ago

## Observation # Summary 
 While collecting logs for [poo#121573](https://progress.opensuse.org/issues/121573) we realized that the journal of the service is different from the minion-log (visible in the webui) which is different from what we expect from reading the code. An example: 

 minion log on webui: 
 ``` 
     [info] [#45742] Cache size of "/var/lib/openqa/cache" is 48 GiB, with limit 50 GiB 
     [info] [#45742] Downloading "autoyast_SLES-12-SP4-ppc64le-HA-updated.qcow2" from "http://openqa.suse.de/tests/10091929/asset/hdd/autoyast_SLES-12-SP4-ppc64le-HA-updated.qcow2" 
     [info] [#45742] Content of "/var/lib/openqa/cache/openqa.suse.de/autoyast_SLES-12-SP4-ppc64le-HA-updated.qcow2" has not changed, updating last use 
 ``` 

 `journalctl -u    openqa-worker-cacheservice-minion.service`: 
 ``` 
 Dec 06 13:29:15 powerqaworker-qam-1 openqa-worker-cacheservice-minion[50194]: [50194] [i] Downloading: "sle-12-SP4-ppc64le-ha-alpha-alpha-node01.qcow2" 
 Dec 06 13:29:52 powerqaworker-qam-1 openqa-worker-cacheservice-minion[50194]: [50194] [i] Cache size of "/var/lib/openqa/cache" is 50 GiB, with limit 50 GiB 
 ``` 

 @okurz also found some code which should print "purging" messages when cached assets are deleted. These I can't find at all in the journal. 

 It also seems that just sometimes the minion-id is logged in the journal which makes it quite hard to find the corresponding minion job from the journal and vice versa. 

 ## Acceptance criteria 
 * **AC1:** Log output on the webui of a minion is consistent and the same as in the system journal 
 * **AC2:** Asset deletion is also logged inside the system journal 
 * **AC3:** System journal includes a reference to the according minion job id 

 ## Suggestions 
 * Try to understand why #121573 https://progress.opensuse.org/issues/121573 went wrong and take a look at several logs we have for the cacheservice. Observe how all these logfiles are telling different things and it is quite hard to link them together to create a complete picture of what happens at that time with the job and cacheservice.

Back