Project

General

Profile

action #116596

CircleCI not reporting subtests anymore size:M

Added by tinita 2 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Organisational
Target version:
Start date:
2022-09-15
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

Only the test is reported back to Github, and api and ui are pending endlessly:
https://github.com/os-autoinst/openQA/pull/4804

test Successful in 21m — Workflow: test
ci/circleci: api Expected — Waiting for status to be reported
Required
ci/circleci: ui Expected — Waiting for status to be reported
Required

Acceptance criteria

  • AC1: All common CI tests in openQA are properly reported in github pull requests

Suggestions

  • Research if there was anything that changed lately for circleCI
  • Ask in some circleCI support chat or something

Workaround

Look into the details of the pipeline and if all tests pass then manually merge

circleci-simple-pr.png (41.7 KB) circleci-simple-pr.png tinita, 2022-09-28 14:07
github-commit-checks.png (78.9 KB) github-commit-checks.png tinita, 2022-09-28 14:35
13855
13858

Related issues

Related to openQA Project - action #116593: Dependency cron openqabot needs to be recovered or replaced size:MResolved2022-09-15

History

#1 Updated by tinita 2 months ago

  • Description updated (diff)

#2 Updated by okurz 2 months ago

  • Related to action #116593: Dependency cron openqabot needs to be recovered or replaced size:M added

#3 Updated by okurz 2 months ago

  • Subject changed from CircleCI not reporting subtests anymore to CircleCI not reporting subtests anymore size:M
  • Description updated (diff)
  • Status changed from New to Workable
  • Priority changed from Normal to High

#4 Updated by mkittler 2 months ago

Looks like all checks are actually reported back on https://github.com/os-autoinst/openQA/pull/4808/checks?check_run_id=8447819964. Only on the PR page under "Some checks were not successful" the checks ci/circleci: api and ci/circleci: ui are not reporting back. Other checks aren't even shown at all (shouldn't e.g. ci/circleci: checkstyle be there?).

Or is it maybe normal that only test (the name of the whole workflow) is shown anymore? But then it is strange that some of the individual jobs are still shown (even if not reporting back).

#5 Updated by mkittler 2 months ago

Some older PRs like https://github.com/os-autoinst/openQA/pull/4778 show only checks for the individual jobs and the overall workflow test is not present at all in the checklist. So I suppose that's how it is supposed to look like.

Maybe we can look into it in the next mob session.

And if you don't have a clue we can still ask for help via https://support.circleci.com/hc/en-us/requests/new?ticket_form_id=855268 or https://discuss.circleci.com (also see their general support page: https://support.circleci.com/hc/en-us).

#6 Updated by okurz 2 months ago

  • Project changed from openQA Infrastructure to openQA Project
  • Category set to Organisational

#7 Updated by tinita 2 months ago

  • Status changed from Workable to In Progress
  • Assignee set to tinita

#8 Updated by tinita 2 months ago

13855

As a first step I made a very simple test:

All three tests are showing, and not the workflow name itself (which would be say-hello-workflow)
See screenshot.

So in principle it should work, but something is different for our setup.

#9 Updated by tinita 2 months ago

13858

I noticed that when clicking on the commit checks (that little icon (check mark, orance circle, red cross) besides the sha in gihub web ui, then all checks are reported as expected. See screenshot.

#10 Updated by tinita about 2 months ago

  • Status changed from In Progress to Feedback

I got it working again, by enabling GitHub Status Updates in the Advanced Project Settings.
What I couldn't find out is, why this suddenly stopped working.

The features "GitHub Status Updates" and "GitHub checks" are explained here: https://circleci.com/docs/enable-checks

Apparently "GitHub Checks" reports the CircleCI workflow with a link to the github checks tab, and links to CircleCI from there to the jobs, while "GitHub Status Updates" reports each CircleCI job directly in the PR checks box.

The second question was, why were the ui and api check so special? The individual jobs weren't reported anymore, only the test workflow, but ui and api at least were still appearing as pending. And that is because in the openQA settings / branch protection rules, someone configured that api and ui must pass before anyone can merge. That's a GitHub feature additionally to the mergify.yml config.

Here is a PR showing all checks: https://github.com/os-autoinst/openQA/pull/4825

#11 Updated by okurz about 2 months ago

I can agree with your confusion :) Great that you could fix it!

#12 Updated by tinita about 2 months ago

  • Status changed from Feedback to Resolved

Also available in: Atom PDF