Project

General

Profile

Actions

action #163187

closed

Uploading coverage to Codecov via the CircleCI setup of openQA no longer works size:M

Added by mkittler 3 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2024-07-03
Due date:
2024-07-23
% Done:

0%

Estimated time:

Description

It runs into a 401 error, currently most of the time:

./"codecov" -v create-commit -t <redacted>
info - 2024-06-25 15:25:46,820 -- ci service found: circleci
debug - 2024-06-25 15:25:46,822 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-06-25 15:25:46,824 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-06-25 15:25:46,825 -- Loading config from /home/squamata/project/codecov.yml
debug - 2024-06-25 15:25:46,830 -- Starting create commit process --- {"commit_sha": "d100e336909c09e6cb8fe51e6c92488610634411", "parent_sha": null, "pr": "5723", "branch": "pull/5723", "slug": "os-autoinst/openQA", "token": "", "service": "github", "enterprise_url": null}
info - 2024-06-25 15:25:46,898 -- Process Commit creating complete
debug - 2024-06-25 15:25:46,899 -- Commit creating result --- {"result": "RequestResult(error=RequestError(code='HTTP Error 401', params={}, description='{\"detail\":\"Invalid token header. No credentials provided.\"}'), warnings=[], status_code=401, text='{\"detail\":\"Invalid token header. No credentials provided.\"}')"}
error - 2024-06-25 15:25:46,899 -- Commit creating failed: {"detail":"Invalid token header. No credentials provided."}
./"codecov" -v create-report -t <redacted>
info - 2024-06-25 15:25:47,778 -- ci service found: circleci
debug - 2024-06-25 15:25:47,779 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-06-25 15:25:47,781 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-06-25 15:25:47,782 -- Loading config from /home/squamata/project/codecov.yml
debug - 2024-06-25 15:25:47,788 -- Starting create report process --- {"commit_sha": "d100e336909c09e6cb8fe51e6c92488610634411", "code": "default", "slug": "os-autoinst/openQA", "service": "github", "enterprise_url": null, "token": ""}
info - 2024-06-25 15:25:47,826 -- Process Report creating complete
debug - 2024-06-25 15:25:47,826 -- Report creating result --- {"result": "RequestResult(error=RequestError(code='HTTP Error 401', params={}, description='{\"detail\":\"Invalid token header. No credentials provided.\"}'), warnings=[], status_code=401, text='{\"detail\":\"Invalid token header. No credentials provided.\"}')"}
error - 2024-06-25 15:25:47,826 -- Report creating failed: {"detail":"Invalid token header. No credentials provided."}
./codecov -v do-upload -t <redacted> -n "130467"   -f cover_db/codecov.json
info - 2024-06-25 15:25:48,692 -- ci service found: circleci
debug - 2024-06-25 15:25:48,693 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-06-25 15:25:48,695 -- versioning system found: <class 'codecov_cli.helpers.versioning_systems.GitVersioningSystem'>
debug - 2024-06-25 15:25:48,696 -- Loading config from /home/squamata/project/codecov.yml
debug - 2024-06-25 15:25:48,702 -- Starting upload processing --- {"branch": "pull/5723", "build_code": "130467", "build_url": "https://circleci.com/gh/os-autoinst/openQA/130467", "commit_sha": "d100e336909c09e6cb8fe51e6c92488610634411", "disable_file_fixes": false, "disable_search": false, "enterprise_url": null, "env_vars": {}, "files_search_exclude_folders": [], "files_search_explicitly_listed_files": ["cover_db/codecov.json"], "files_search_root_folder": "/home/squamata/project", "flags": [], "git_service": "github", "handle_no_reports_found": false, "job_code": "0", "name": "130467", "network_filter": null, "network_prefix": null, "network_root_folder": "/home/squamata/project", "plugin_names": ["xcode", "gcov", "pycoverage"], "pull_request_number": "5723", "report_code": "default", "slug": "os-autoinst/openQA", "token": "", "upload_file_type": "coverage"}
debug - 2024-06-25 15:25:48,702 -- Selected preparation plugins --- {"selected_plugins": ["<class 'codecov_cli.plugins.xcode.XcodePlugin'>", "<class 'codecov_cli.plugins.gcov.GcovPlugin'>", "<class 'codecov_cli.plugins.pycoverage.Pycoverage'>"]}
debug - 2024-06-25 15:25:48,702 -- Running preparation plugin: <class 'codecov_cli.plugins.xcode.XcodePlugin'>
debug - 2024-06-25 15:25:48,702 -- Running xcode plugin...
warning - 2024-06-25 15:25:48,702 -- xcrun is not installed or can't be found.
debug - 2024-06-25 15:25:48,702 -- Running preparation plugin: <class 'codecov_cli.plugins.gcov.GcovPlugin'>
debug - 2024-06-25 15:25:48,702 -- Running gcov plugin...
warning - 2024-06-25 15:25:48,822 -- No gcov data found.
debug - 2024-06-25 15:25:48,822 -- Running preparation plugin: <class 'codecov_cli.plugins.pycoverage.Pycoverage'>
warning - 2024-06-25 15:25:48,822 -- coverage.py is not installed or can't be found.
debug - 2024-06-25 15:25:48,823 -- Collecting relevant files
info - 2024-06-25 15:25:49,135 -- Found 6 coverage files to report
info - 2024-06-25 15:25:49,135 -- > /home/squamata/project/t/data/openqa-trigger-from-obs/BatchedProj/Batch1/.run_191216_150610/Media1_ftp_ftp.lst
info - 2024-06-25 15:25:49,135 -- > /home/squamata/project/t/data/openqa-trigger-from-obs/Proj1/.run_190703_143010_469.1/files_iso.lst
info - 2024-06-25 15:25:49,135 -- > /home/squamata/project/t/data/openqa-trigger-from-obs/BatchedProj/Batch2/.run_191216_150610/files_iso.lst
info - 2024-06-25 15:25:49,135 -- > /home/squamata/project/cover_db/codecov.json
info - 2024-06-25 15:25:49,135 -- > /home/squamata/project/t/data/openqa-trigger-from-obs/BatchedProj/Batch1/.run_191216_150610/files_iso.lst
info - 2024-06-25 15:25:49,135 -- > /home/squamata/project/t/data/openqa-trigger-from-obs/Proj1/.run_190701_143010_468.2/files_iso.lst
debug - 2024-06-25 15:25:49,163 -- Selected uploader to use: <class 'codecov_cli.services.upload.upload_sender.UploadSender'>
debug - 2024-06-25 15:25:49,173 -- Sending upload request to Codecov
info - 2024-06-25 15:25:49,212 -- Process Upload complete
debug - 2024-06-25 15:25:49,212 -- Upload result --- {"result": "RequestResult(error=RequestError(code='HTTP Error 401', params={}, description='{\"detail\":\"Invalid token header. No credentials provided.\"}'), warnings=[], status_code=401, text='{\"detail\":\"Invalid token header. No credentials provided.\"}')"}
error - 2024-06-25 15:25:49,212 -- Upload failed: {"detail":"Invalid token header. No credentials provided."}

It is still considered successful but the coverage report/statuses are missing.

Acceptance criteria

  • AC1: Coverage checks in the openQA-CI work

Suggestions

Workaround

Look at the coverage report uploaded as CircleCI artifact and merge PRs manually if it looks good.

Actions #1

Updated by tinita 3 months ago

  • Description updated (diff)
Actions #2

Updated by okurz 3 months ago

  • Tags set to reactive work, CI, codecov, circleci
  • Target version set to Ready
Actions #3

Updated by okurz 3 months ago

  • Subject changed from Uploading coverage to Codecov via the CircleCI setup of openQA no longer works to Uploading coverage to Codecov via the CircleCI setup of openQA no longer works size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #4

Updated by okurz 3 months ago

  • Description updated (diff)
Actions #5

Updated by mkittler 3 months ago

  • Description updated (diff)
  • Status changed from Workable to In Progress
  • Assignee set to mkittler
Actions #6

Updated by mkittler 3 months ago

  • Description updated (diff)
Actions #7

Updated by mkittler 3 months ago

If I download the binary that is also used within CircleCI and run it locally with the same params as on CircleCI (/hdd/downloads/codecov -v create-commit -t …) I don't run into the error we get within CircleCI. Maybe the token in CircleCI is not correct after all (not sure as it is redacted).

Actions #8

Updated by mkittler 3 months ago

Maybe it helps to specify the context in which the token is configured as documented on https://circleci.com/docs/env-vars. PR: https://github.com/os-autoinst/openQA/pull/5750
If that helps the token was never required before but now it is.

I also had a look on https://circleci.com/developer/orbs/orb/codecov/codecov?version=4.1.0 but couldn't find a mistake in our usage of the orb. The orb also hasn't changed much.

Note that OBS doesn't use the orb. They just download/run the uploader manually, see https://github.com/openSUSE/open-build-service/blob/d29bd63d406582221d69dd5938a19ee7534bd674/.circleci/conditional_config.yml#L377. They don't configure the token anywhere so I have no idea how the OBS setup works if the token is really required now (which might be the case as mentioned in the first paragraph).

Actions #9

Updated by openqa_review 3 months ago

  • Due date set to 2024-07-23

Setting due date based on mean cycle time of SUSE QE Tools

Actions #11

Updated by mkittler 3 months ago

Actions #12

Updated by mkittler 3 months ago

  • Status changed from In Progress to Feedback
Actions #13

Updated by okurz 3 months ago

approved. Do you think we can improve the error reporting to not silently accept and then fail on empty/inaccessible tokens?

Actions #14

Updated by mkittler 3 months ago

Do you think we can improve the error reporting to not silently accept and then fail on empty/inaccessible tokens?

For this we would probably need to parse the output of codecov (the uploader binary). It unfortunately returns 0 in the error case (at least in this error case). I verified this locally. According to its --help it also has no flag to change that behavior. To parse the output we would probably need to invoke codecov directly (not using the Orb anymore). I think that's not worth it.

Actions #15

Updated by mkittler 3 months ago

I created https://github.com/os-autoinst/openQA/pull/5753 so despite the new setting the GitHub token won't be accessible from forked repos. For this I created a new context and restricted it to the tools team and our bot. I guess I'll temporarily remove the tools team and see whether the token is actually not present anymore on a PR from my fork.

If everything works as expected I'd consider this ticket resolved because improving the codecov upload further is probably not worth it.

Actions #16

Updated by mkittler 3 months ago

  • Status changed from Feedback to Resolved

It didn't work because the user "Scheduled" (which is used by CircleCI to invoke the nightly job) is blocked by the context, see https://app.circleci.com/pipelines/github/os-autoinst/openQA/13977/workflows/a12d1b3b-6f54-4560-b081-bc0a5950e949.

So the blocking actually works but goes too far. I just somehow expected that the CircleCI user has some special role and it would work there regardless. There seems to be no way to allow the "Scheduled" user as this is only about our own teams we define in our GitHub orga. So I added "All members" back and instead restricted the context by branch. The relevant branch is "master". This seems to work now. I re-triggered the existing job (which means the associated user is still "Scheduled" and not me) and it works, see https://app.circleci.com/pipelines/github/os-autoinst/openQA/13977/workflows/372fd2ac-b5f7-4d45-9b5b-16b85c42bfda.

This shows that the context is effective so I'm confident it'll work when one of those CI jobs actually need to create a PR. So I'm resolving this issue for now.

Actions

Also available in: Atom PDF