action #116596
closedCircleCI not reporting subtests anymore size:M
Description
Observation¶
Only the test is reported back to Github, and api and ui are pending endlessly:
https://github.com/os-autoinst/openQA/pull/4804
test Successful in 21m — Workflow: test
ci/circleci: api Expected — Waiting for status to be reported
Required
ci/circleci: ui Expected — Waiting for status to be reported
Required
Acceptance criteria¶
- AC1: All common CI tests in openQA are properly reported in github pull requests
Suggestions¶
- Research if there was anything that changed lately for circleCI
- Ask in some circleCI support chat or something
Workaround¶
Look into the details of the pipeline and if all tests pass then manually merge
Files
| circleci-simple-pr.png (41.7 KB) circleci-simple-pr.png | |||
| github-commit-checks.png (78.9 KB) github-commit-checks.png |
Updated by okurz over 2 years ago
- Related to action #116593: Dependency cron openqabot needs to be recovered or replaced size:M added
Updated by okurz over 2 years ago
- Subject changed from CircleCI not reporting subtests anymore to CircleCI not reporting subtests anymore size:M
- Description updated (diff)
- Status changed from New to Workable
- Priority changed from Normal to High
Updated by mkittler about 2 years ago
Looks like all checks are actually reported back on https://github.com/os-autoinst/openQA/pull/4808/checks?check_run_id=8447819964. Only on the PR page under "Some checks were not successful" the checks ci/circleci: api and ci/circleci: ui are not reporting back. Other checks aren't even shown at all (shouldn't e.g. ci/circleci: checkstyle be there?).
Or is it maybe normal that only test (the name of the whole workflow) is shown anymore? But then it is strange that some of the individual jobs are still shown (even if not reporting back).
Updated by mkittler about 2 years ago
Some older PRs like https://github.com/os-autoinst/openQA/pull/4778 show only checks for the individual jobs and the overall workflow test is not present at all in the checklist. So I suppose that's how it is supposed to look like.
Maybe we can look into it in the next mob session.
And if you don't have a clue we can still ask for help via https://support.circleci.com/hc/en-us/requests/new?ticket_form_id=855268 or https://discuss.circleci.com (also see their general support page: https://support.circleci.com/hc/en-us).
Updated by okurz about 2 years ago
- Project changed from openQA Infrastructure (public) to openQA Project (public)
- Category set to Organisational
Updated by tinita about 2 years ago
- Status changed from Workable to In Progress
- Assignee set to tinita
Updated by tinita about 2 years ago
- File circleci-simple-pr.png circleci-simple-pr.png added
As a first step I made a very simple test:
- Created a repo https://github.com/perlpunk/test-for-circleci
- Set up a very simple workflow where the
finaltest depends ontest1andtest2: https://github.com/perlpunk/test-for-circleci/blob/main/.circleci/config.yml - Ran a PR: https://github.com/perlpunk/test-for-circleci/pull/1
All three tests are showing, and not the workflow name itself (which would be say-hello-workflow)
See screenshot.
So in principle it should work, but something is different for our setup.
Updated by tinita about 2 years ago
- File github-commit-checks.png github-commit-checks.png added
I noticed that when clicking on the commit checks (that little icon (check mark, orance circle, red cross) besides the sha in gihub web ui, then all checks are reported as expected. See screenshot.
Updated by tinita about 2 years ago
- Status changed from In Progress to Feedback
I got it working again, by enabling GitHub Status Updates in the Advanced Project Settings.
What I couldn't find out is, why this suddenly stopped working.
The features "GitHub Status Updates" and "GitHub checks" are explained here: https://circleci.com/docs/enable-checks
Apparently "GitHub Checks" reports the CircleCI workflow with a link to the github checks tab, and links to CircleCI from there to the jobs, while "GitHub Status Updates" reports each CircleCI job directly in the PR checks box.
The second question was, why were the ui and api check so special? The individual jobs weren't reported anymore, only the test workflow, but ui and api at least were still appearing as pending. And that is because in the openQA settings / branch protection rules, someone configured that api and ui must pass before anyone can merge. That's a GitHub feature additionally to the mergify.yml config.
Here is a PR showing all checks: https://github.com/os-autoinst/openQA/pull/4825
Updated by okurz about 2 years ago
I can agree with your confusion :) Great that you could fix it!