Project

General

Profile

Actions

action #130258

closed

[openqa_logwarn] git warning: exhaustive rename detection (investigation tab) size:M

Added by tinita 12 months ago. Updated 11 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
Due date:
2023-06-22
% Done:

0%

Estimated time:

Description

Observation

We have these warnings in our logs, very rarely though:

[2023-05-03T08:40:57.434993Z] [warn] [pid:14091]  42-inst-oninstallation-uefi-20150922.json          |  16 -----
 42-inst-oninstallation-uefi-20150922.png           | Bin 132801 -> 0 bytes
 ...tallation_overview-Staging_Update-20180208.json |  16 -----
 ...stallation_overview-Staging_Update-20180208.png | Bin 48575 -> 0 bytes
 DIALOG-packages-notifications-20190105.json        |  15 -----
 DIALOG-packages-notifications-20190105.png         | ...
...
 zypper-zdup-finish-20201012.png                    | Bin 4912 -> 0 bytes
  14597 files changed, 12160 insertions(+), 139070 deletions(-)
warning: exhaustive rename detection was skipped due to too many files.
warning: you may want to set your diff.renameLimit variable to at least 13439 and retry the command.Bin 67666 -> 0 bytes

This apparently comes from
https://openqa.opensuse.org/tests/3296915#investigation

Acceptance criteria

  • AC1: needles_diff_stat is not slowing down the investigation tab
  • AC2: We are not alerted about those git warnings

Suggestions

  • See #110677 for a related fix for git_log_diff and comments about what to do about needles_diff_stat
  • Use output from git rev-list --count to decide if the needle diff stat should be rendered at all
  • Use a configurable limit (compare to git log limit of 200)

Out of scope

  • Point to a remote diff, e.g. on github

Related issues 1 (0 open1 closed)

Copied from openQA Project - action #128591: [openqa_logwarn] logwarn reports the same entry over and over size:MResolvedtinita2023-05-03

Actions
Actions #1

Updated by tinita 12 months ago

  • Copied from action #128591: [openqa_logwarn] logwarn reports the same entry over and over size:M added
Actions #2

Updated by tinita 12 months ago

  • Description updated (diff)
Actions #3

Updated by okurz 12 months ago

  • Priority changed from Normal to High
Actions #4

Updated by tinita 12 months ago

  • Status changed from New to In Progress
  • Assignee set to tinita
Actions #5

Updated by openqa_review 12 months ago

  • Due date set to 2023-06-22

Setting due date based on mean cycle time of SUSE QE Tools

Actions #6

Updated by tinita 12 months ago

To reproduce the warning, go into os-autoinst-needles-opensuse and do:

git diff --stat 1b21ed87c..bd6156b9c

It's not only about the warning, but this can also take long. It takes over 30s on o3, while it's much faster (3s) on my laptop.

Looking at the number of commits:

% git rev-list --count 1b21ed87c..bd6156b9c
2937

it's clear that people usually wouldn't want to look in such a huge diff anyway.
At least it's a big waste of time to generate this every time someone goes to the investigation tab (and probably never looks at it).

The counting of commits is relatively fast, so I would now do the rev-list --count command and just show "Too many commits" if the number is higher than the value we already configured for the git log.

Actions #7

Updated by okurz 12 months ago

  • Subject changed from [openqa_logwarn] git warning: exhaustive rename detection (investigation tab) to [openqa_logwarn] git warning: exhaustive rename detection (investigation tab) size:M
  • Description updated (diff)
Actions #8

Updated by tinita 12 months ago

After stumbling over various unrelated bugs, finally:
https://github.com/os-autoinst/openQA/pull/5205 Limit git diff to a maximum number of commits

Actions #9

Updated by tinita 12 months ago

  • Status changed from In Progress to Feedback
Actions #10

Updated by tinita 11 months ago

https://github.com/os-autoinst/openQA/pull/5205 merged, will check logs after deployment

Actions #11

Updated by tinita 11 months ago

  • Status changed from Feedback to Resolved

I can see request times up to 10 seconds in rare cases now for investigation_ajax, but that's it; the longer taking requests are gone now, and I don't expect such messages from logwarn any more.

Actions

Also available in: Atom PDF