action #130258
closed[openqa_logwarn] git warning: exhaustive rename detection (investigation tab) size:M
Description
Observation¶
We have these warnings in our logs, very rarely though:
[2023-05-03T08:40:57.434993Z] [warn] [pid:14091] 42-inst-oninstallation-uefi-20150922.json | 16 -----
42-inst-oninstallation-uefi-20150922.png | Bin 132801 -> 0 bytes
...tallation_overview-Staging_Update-20180208.json | 16 -----
...stallation_overview-Staging_Update-20180208.png | Bin 48575 -> 0 bytes
DIALOG-packages-notifications-20190105.json | 15 -----
DIALOG-packages-notifications-20190105.png | ...
...
zypper-zdup-finish-20201012.png | Bin 4912 -> 0 bytes
14597 files changed, 12160 insertions(+), 139070 deletions(-)
warning: exhaustive rename detection was skipped due to too many files.
warning: you may want to set your diff.renameLimit variable to at least 13439 and retry the command.Bin 67666 -> 0 bytes
This apparently comes from
https://openqa.opensuse.org/tests/3296915#investigation
Acceptance criteria¶
- AC1:
needles_diff_stat
is not slowing down the investigation tab - AC2: We are not alerted about those git warnings
Suggestions¶
- See #110677 for a related fix for
git_log_diff
and comments about what to do aboutneedles_diff_stat
- Use output from
git rev-list --count
to decide if the needle diff stat should be rendered at all - Use a configurable limit (compare to git log limit of 200)
Out of scope¶
- Point to a remote diff, e.g. on github
Updated by tinita over 1 year ago
- Copied from action #128591: [openqa_logwarn] logwarn reports the same entry over and over size:M added
Updated by tinita over 1 year ago
- Status changed from New to In Progress
- Assignee set to tinita
Updated by openqa_review over 1 year ago
- Due date set to 2023-06-22
Setting due date based on mean cycle time of SUSE QE Tools
Updated by tinita over 1 year ago
To reproduce the warning, go into os-autoinst-needles-opensuse and do:
git diff --stat 1b21ed87c..bd6156b9c
It's not only about the warning, but this can also take long. It takes over 30s on o3, while it's much faster (3s) on my laptop.
Looking at the number of commits:
% git rev-list --count 1b21ed87c..bd6156b9c
2937
it's clear that people usually wouldn't want to look in such a huge diff anyway.
At least it's a big waste of time to generate this every time someone goes to the investigation tab (and probably never looks at it).
The counting of commits is relatively fast, so I would now do the rev-list --count
command and just show "Too many commits" if the number is higher than the value we already configured for the git log.
Updated by okurz over 1 year ago
- Subject changed from [openqa_logwarn] git warning: exhaustive rename detection (investigation tab) to [openqa_logwarn] git warning: exhaustive rename detection (investigation tab) size:M
- Description updated (diff)
Updated by tinita over 1 year ago
After stumbling over various unrelated bugs, finally:
https://github.com/os-autoinst/openQA/pull/5205 Limit git diff to a maximum number of commits
Updated by tinita over 1 year ago
https://github.com/os-autoinst/openQA/pull/5205 merged, will check logs after deployment
Updated by tinita over 1 year ago
- Status changed from Feedback to Resolved
I can see request times up to 10 seconds in rare cases now for investigation_ajax, but that's it; the longer taking requests are gone now, and I don't expect such messages from logwarn any more.