Project

General

Profile

Actions

action #53546

open

Easier dependencies handling for packages, e.g. reduce duplication of build requirements in spec, documentation, Dockerfile

Added by okurz almost 5 years ago. Updated about 3 years ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2019-06-27
Due date:
% Done:

0%

Estimated time:

Description

Motivation

For both os-autoinst and openQA "build requirements" are defined in multiple places, e.g. https://github.com/os-autoinst/openQA/blob/master/docker/travis_test/Dockerfile#L10 , https://github.com/os-autoinst/openQA/blob/master/cpanfile , https://github.com/os-autoinst/openQA/blob/master/openQA.spec#L37 and correspondingly for os-autoinst . This fact and also the current setup based on the Dockerfile "docker/travis_test/Dockerfile" which a container is built from in OBS, but only after merge of a PR, makes it hard to understand and hard to introduce new dependencies and test for them properly.


Related issues 6 (1 open5 closed)

Related to openQA Project - action #43619: Improve workflow for dealing with openQA's dependenciesResolvedtinita2018-11-09

Actions
Related to openQA Project - action #55346: packaging test as part of every PRResolvedcoolo2019-08-11

Actions
Related to openQA Project - action #66649: Test GitHub actions for os-autoinstResolvedtinita2020-05-08

Actions
Related to openQA Project - action #66721: Use GitHub actions for os-autoinstResolvedokurz2020-05-12

Actions
Related to openQA Project - action #73309: every time a direct dependency is updated in Factory our CI jobs fail until the package is updatedWorkable2020-10-13

Actions
Blocked by openQA Project - action #56525: Create rpm requires from cpanfileResolvedtinita2019-09-06

Actions
Actions #1

Updated by okurz almost 5 years ago

What I thought of and partially tried:

  • Define a "devel" package in the spec file and let it require all build time dependencies which are needed also for use outside OBS and install that virtual package whenever we want to develop, e.g. as in https://github.com/os-autoinst/os-autoinst/pull/1171
  • Generate the spec file completely from a template and generated information, e.g. as with cpan2spec. Challenge would be where to store the spec-file. The best I could think of is to use travis CI to generate the spec file for every merge to master and save it in a special non-master git branch which the OBS service can check out. Why not store in master? A: To not mix generated and human-written files
  • Extract the dependencies for the container scripts from the spec file, e.g. by parsing it and transforming into a list to give to zypper on-the-fly

All of the above help to define a single place for dependencies. Now to actually solve the problem that dependencies would only be active after merge not before one could use the same dependency list in the downstream "openqa" container (not "travis_test" aka. "openqa_dev") and call zypper -n in $deps again to cover any potential diff in packages, e.g. when adding new dependencies.

Other thing: We have a container "travis_test" aka. "openqa_dev" which is built and published by OBS and a container "openqa" based on openqa_dev which is built and run only during travis CI tests. This is confusing (the names in particular) and not helping us to cover testing of the actual other Dockerfiles we have. Maybe we can get rid of "openqa" by building "openqa_dev" again but based on cached results? Maybe define the "openqa" container just as here-doc and a less complicated script with more setup-helper-scripts, e.g. as in https://github.com/okurz/openQA/commit/b255b05939417792580abf8d5e017c28d33d75b1 where I reduce duplication to setup a test database

EDIT: Maybe https://stackoverflow.com/questions/4339417/is-there-any-syntax-or-trick-to-be-able-to-create-a-multiline-rpm-spec-file-macr also helps to define lists of dependencies and include in spec file.

Actions #2

Updated by okurz almost 5 years ago

  • Status changed from New to Feedback
  • Assignee set to okurz
Actions #3

Updated by okurz almost 5 years ago

  • Related to action #43619: Improve workflow for dealing with openQA's dependencies added
Actions #4

Updated by okurz over 4 years ago

  • Blocked by action #56525: Create rpm requires from cpanfile added
Actions #5

Updated by okurz over 4 years ago

  • Related to action #55346: packaging test as part of every PR added
Actions #6

Updated by okurz over 4 years ago

  • Status changed from Feedback to Blocked

waiting for #56525 first

Actions #7

Updated by okurz almost 4 years ago

  • Related to action #66649: Test GitHub actions for os-autoinst added
Actions #8

Updated by okurz almost 4 years ago

Discussed with tinita and mkittler and we re-confirmed what I wrote in #53546#note-1 , there is one noteworthy difference. The openqa_dev container definition is now only used within os-autoinst but stored in the openQA repo. That one can be moved to os-autoinst instead.

Actions #9

Updated by tinita almost 4 years ago

  • Blocked by action #66721: Use GitHub actions for os-autoinst added
Actions #10

Updated by tinita almost 4 years ago

  • Blocked by deleted (action #66721: Use GitHub actions for os-autoinst)
Actions #11

Updated by tinita almost 4 years ago

  • Related to action #66721: Use GitHub actions for os-autoinst added
Actions #12

Updated by tinita almost 4 years ago

  • Related to deleted (action #66721: Use GitHub actions for os-autoinst)
Actions #13

Updated by tinita almost 4 years ago

  • Related to action #66721: Use GitHub actions for os-autoinst added
Actions #14

Updated by okurz over 3 years ago

  • Target version set to Ready
Actions #15

Updated by okurz over 3 years ago

@mkittler @tinita see #53546#note-8 , we still have both "openqa_dev" and "os-autoinst_dev" in https://build.opensuse.org/project/show/devel:openQA , can we delete the first or something?

Actions #16

Updated by tinita over 3 years ago

Yes, I think https://build.opensuse.org/package/show/devel:openQA/openqa_dev can now be deleted

Edit: actually it is still used as the documented way to run tests locally.

Actions #17

Updated by tinita over 3 years ago

PR to remove Dockerfile in openQA: https://github.com/os-autoinst/openQA/pull/3372

Actions #18

Updated by okurz over 3 years ago

  • Related to action #73309: every time a direct dependency is updated in Factory our CI jobs fail until the package is updated added
Actions #19

Updated by okurz about 3 years ago

  • Status changed from Blocked to New
  • Assignee deleted (okurz)
  • Priority changed from Normal to Low
  • Target version changed from Ready to future

a major part of making working with dependencies has been done by tinita in #43619 . We don't have a good idea for the other points mentioned. I would still favor a solution that is less openSUSE centric and duplicates less code in our source repo even if it is auto-generated so I will keep the ticket but keep out of our backlog as an idea for later.

Actions

Also available in: Atom PDF