action #104088: bot-ng pipeline consistently fails with KeyError: 'test_issues' - openQA Infrastructure - openSUSE Project Management Tool

Actions

Copy link

action #104088

closed

bot-ng pipeline consistently fails with KeyError: 'test_issues'

Added by livdywan over 2 years ago. Updated over 2 years ago.

Status:

Resolved

Priority:

High

Assignee:

okurz

Category:

Target version:

openQA Project - Ready

Start date:

2021-12-16

Due date:

% Done:

Estimated time:

Description

bot-ng pipeline consistently fails like so:

ERROR: Missing product in /etc/openqabot/bot.yml
ERROR: Missing product in /etc/openqabot/mr.yml
Traceback (most recent call last):
  File "./bot-ng.py", line 5, in <module>
    main()
  File "/builds/qa-maintenance/bot-ng/openqabot/main.py", line 40, in main
    sys.exit(cfg.func(cfg))
  File "/builds/qa-maintenance/bot-ng/openqabot/args.py", line 32, in do_aggregate_schedule
    bot = OpenQABot(args)
  File "/builds/qa-maintenance/bot-ng/openqabot/openqabot.py", line 26, in __init__
    args.configs, args.disable_aggregates, args.disable_incidents, extrasettings
  File "/builds/qa-maintenance/bot-ng/openqabot/loader/config.py", line 47, in load_metadata
    ret.append(Aggregate(data["product"], settings, data[key]))
  File "/builds/qa-maintenance/bot-ng/openqabot/types/aggregate.py", line 29, in __init__
    self.test_issues = self.normalize_repos(config)
  File "/builds/qa-maintenance/bot-ng/openqabot/types/aggregate.py", line 35, in normalize_repos
    for key, value in config["test_issues"].items()
KeyError: 'test_issues'

A re-trigger did not make a difference.

Actions

Copy link

Updated by livdywan over 2 years ago

Target version set to Ready

Actions

Copy link

Updated by okurz over 2 years ago

Priority changed from High to Urgent

Actions

Copy link

Updated by okurz over 2 years ago

but if bot-ng "can't find updates" I still consider the impact of SUSE IT EngInfra maintenance likely

Actions

Copy link

Updated by coolo over 2 years ago

If - but there is no such string in the error

Actions

Copy link

Updated by okurz over 2 years ago

Status changed from New to In Progress
Assignee set to okurz

doing https://gitlab.suse.de/qa-maintenance/metadata/-/merge_requests/555 with a revert of https://gitlab.suse.de/qa-maintenance/metadata/-/commit/550ee0e22abe415bbca1d1d67c716dc9e8ac9249

As coolo explained:
the aggregate hash has no issues configured, which makes it invalid as without incidents there is nothing to aggregate.
looks like some yaml lint is due on metadata PRs

Actions

Copy link

Updated by okurz over 2 years ago

Status changed from In Progress to Feedback
Priority changed from Urgent to High

I reverted and merged and will retrigger the run in bot-ng.
Now it's the IT maintenance window https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs/747709#L17

fatal: unable to access 'https://gitlab.suse.de/qa-maintenance/bot-ng.git/': Could not resolve host: gitlab.suse.de

So I don't want to waste more time with that and we wait until the next day or at least after the SUSE IT maintenance window

Actions

Copy link

Updated by jbaier_cz over 2 years ago

okurz wrote:

fatal: unable to access 'https://gitlab.suse.de/qa-maintenance/bot-ng.git/': Could not resolve host: gitlab.suse.de
So I don't want to waste more time with that and we wait until the next day or at least after the SUSE IT maintenance window

That is unfortunately not a maintenance window, that is some sort of known problem inside the already deprecated CAASP cluster where the runners are. A ticket for that exists somewhere (saw it in the past)...

Actions

Copy link

Updated by okurz over 2 years ago

Yes, you are right. In the meantime another retrigger worked ok. I assume the problem is fixed. I asked pdostal and fniederwanger if they could include YAML lint checks in the repository to prevent a similar situation in the future.

I also created https://gitlab.suse.de/qa-maintenance/bot-ng/-/merge_requests/43 for an explicit error and suggestion in such situation

Actions

Copy link

Updated by jbaier_cz over 2 years ago

okurz wrote:

I asked pdostal and fniederwanger if they could include YAML lint checks in the repository to prevent a similar situation in the future.

There actually are some checks in that repository already, for example https://gitlab.suse.de/qa-maintenance/metadata/-/blob/master/t/bot.t Unfortunately, it seems it is limited to the old directory only. We definitely need to extend those tests and if we have expected attributes in the metadata we should do a scheme validation.

Actions

Copy link

#10