Project

General

Profile

Actions

action #98496

closed

nightly circle CI job fails in "Install cached packages" with "rpm: no packages given for install" size:M

Added by okurz over 2 years ago. Updated over 2 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
Category:
Organisational
Target version:
Start date:
2021-09-13
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://app.circleci.com/pipelines/github/os-autoinst/openQA/7659/workflows/cc8ba326-aa47-40a4-b43c-fec12d1704b1/jobs/72308 shows

#!/bin/bash -eo pipefail
set -x
if [ -z "$CIRCLE_WORKFLOW_ID" ]; then
  bash .circleci/build_cache.sh
else
  sudo rpm -i -f $(find /var/cache/zypp/packages/ | grep '.rpm$')
fi

+ '[' -z cc8ba326-aa47-40a4-b43c-fec12d1704b1 ']'
++ find /var/cache/zypp/packages/
++ grep '.rpm$'
+ sudo rpm -i -f
rpm: no packages given for install

Exited with code exit status 1

CircleCI received exit code 1

in a previous step "Restoring cache" there is:

No cache is found for key: v1-R3CxihB_CRf9AaxlDij+JU8Ke6fsRQrgKsZfJNeqcCs=-0ctxsgSE3h8sZFrULdZqhQKTjwrMfWf9GrviDcdOQG8=

Acceptance criteria

  • AC1: CI should fail w/ a clear error message if the cache is empty
Actions #1

Updated by livdywan over 2 years ago

  • Subject changed from nightly circle CI job fails in "Install cached packages" with "rpm: no packages given for install" to nightly circle CI job fails in "Install cached packages" with "rpm: no packages given for install" size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #2

Updated by Xiaojing_liu over 2 years ago

  • Assignee set to Xiaojing_liu
Actions #3

Updated by livdywan over 2 years ago

  • Description updated (diff)

Updated the AC after discussing it with Jane and Oli

Actions #4

Updated by okurz over 2 years ago

We discussed about this and found that my initial request was no good. We use the cached packages also to define our consistent, version specific set of packages that we want to rely on. Resorting to a generic zypper call on errors might work but defeat the purpose as we would not realize if there are problems on saving and loading the cache so eventually we would end up always downloading different versions with zypper on the fly which we wanted to avoid. For now I don't have any better idea than https://github.com/os-autoinst/openQA/pull/4238 for now, it's really not much.

Our next best bet would be to find the job on circleCI that actually failed to properly save the cache and prevent the error the next time.

Actions #5

Updated by Xiaojing_liu over 2 years ago

  • Assignee deleted (Xiaojing_liu)

I have no idea how to find the job that failed to cache. Searched the CI jobs and found nothing. I release this ticket.

Actions #6

Updated by tinita over 2 years ago

The last job in master was this one:
https://app.circleci.com/pipelines/github/os-autoinst/openQA/7646/workflows/96da7b3c-e1c9-4d03-a00e-3f41d4444b8b/jobs/72196

Skipping cache generation, cache already exists for key: v1-R3CxihB_CRf9AaxlDij+JU8Ke6fsRQrgKsZfJNeqcCs=-0ctxsgSE3h8sZFrULdZqhQKTjwrMfWf9GrviDcdOQG8=
Found one created at 2021-08-26 08:47:43 +0000 UTC

Looking into the artifacts, at this point the cache seems to be ok:
https://circle-production-customer-artifacts.s3.amazonaws.com/picard/58f7029dc9e77c000129905e/613b7b5e151b363f96b66961-0-build/artifacts/artifacts/packages.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210927T145305Z&X-Amz-SignedHeaders=host&X-Amz-Expires=60&X-Amz-Credential=AKIAJR3Q6CR467H7Z55A%2F20210927%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=cdb02af443eee56fde9b140dd38e5042c83123f8c0ce94dd221bb37f9427c4f4

/var/cache/zypp/packages/openQA/noarch/perl-Carp-Always-0.16-lp152.2.1.noarch.rpm
/var/cache/zypp/packages/openQA/noarch/perl-DBIx-Class-DeploymentHandler-0.002233-lp152.2.3.noarch.rpm
/var/cache/zypp/packages/openQA/noarch/perl-Devel-Cover-Report-Codecov-0.25-lp152.2.5.noarch.rpm
...

My guess is, since it's a cache, there is no guarantee that it will be kept for a certain time.
So maybe just calling build_cache.sh could help.

Actions #7

Updated by okurz over 2 years ago

  • Status changed from Workable to Rejected
  • Assignee set to okurz

somehow this hasn't happened anymore. So something might have fixed it, maybe it will come back.

Actions

Also available in: Atom PDF