action #128129

Updated by okurz about 1 year ago

## Observation 
 It seems multiple, maybe all, pull requests don't have codecov information. is one example. It seems only openQA fork-based pull requests are affected, not os-autoinst and also not pull requests from within the main repo. example 

 OBS is also affected since some days, see their upstream report on . Problems likely started for us at the same time so either something on the side of codecov or circleCI or github changed. 

 Specific details from failing jobs: 

 [2023-04-21T09:17:36.470Z] ['info'] Pinging Codecov:*******&branch=pull%2F5088&build=105794& 
 [2023-04-21T09:17:36.839Z] ['error'] There was an error running the uploader: Error uploading to Error: There was an error fetching the storage URL during POST: 404 - {'detail': ErrorDetail(string='Commit sha does not match Circle build. Please upload with the Codecov repository upload token to resolve issue.', code='not_found')} 
 CircleCI received exit code 0 

 We have a circleCI token in the main repo but that is not accessible to forks. For os-autoinst fork-based PRs without access to a token also still seem to work where we use the github action that works. 

 ## Steps to reproduce 
 Likely any pull request created will not have codecov check results recorded 

 ## Expected result 
 1. Pull requests for os-autoinst+openQA should have code coverage codecov information 
 2. PRs like "dependency cron" are merged automatically by mergify again is a working example from a recent origin-based PR after the problem happened. 

 ## Suggestions 
 * *DONE* Research about the problem if others are hit as well 
 * *DONE* Try to reproduce the problem 
 * Different alternatives we can follow: 

     1. Continue investigation to find a workaround as is not progressing. Maybe we find other github projects also using circleCI+codecov and they might have found better workarounds 

      * For this trying with "circleCI orb": 

     2. Enable the use of codeclimate additionally to codecov, same as OBS did. Maybe also coveralls 

     3. Migrate from circleCI to github action – at least to crosscheck 

 ## Further details 
 See #127688, maybe related? - most likely not, see #12812#note-1 

 ## Out of scope 
 Instead of relying on any external coverage service we could just implement our own coverage target check as we have all the data already uploaded as artifacts. We could add a comment on each pull request pointing to the artifact documents. 

 ## Workaround 
 Create a pull request from the main repo as we have a token there.