QA (public) » openQA Project (public)

On https://build.opensuse.org/projects/devel:openQA/packages/openQA/files/openQA.changes?expand=1 the last update is from "Wed Nov 20 17:14:36 UTC 2024 - okurz@suse.com" so the regular changelog file lacks new changelog entries entirely. (https://build.opensuse.org/projects/devel:openQA/packages/openQA/files/_service:obs_scm:openQA.changes?expand=1 shows the problem as mentioned in the ticket description.)

On https://build.opensuse.org/projects/devel:openQA:tested/packages/openQA/files/openQA.changes?expand=1 the last update is from "Fri Mar 28 16:14:24 UTC 2025 - okurz@suse.com" after "Thu Mar 27 22:15:30 UTC 2025 - okurz@suse.com" so this package actually works as expected.

mkittler wrote in #note-4:

On https://build.opensuse.org/projects/devel:openQA/packages/openQA/files/openQA.changes?expand=1 the last update is from "Wed Nov 20 17:14:36 UTC 2024 - okurz@suse.com" so the regular changelog file lacks new changelog entries entirely. (https://build.opensuse.org/projects/devel:openQA/packages/openQA/files/_service:obs_scm:openQA.changes?expand=1 shows the problem as mentioned in the ticket description.)

On https://build.opensuse.org/projects/devel:openQA:tested/packages/openQA/files/openQA.changes?expand=1 the last update is from "Fri Mar 28 16:14:24 UTC 2025 - okurz@suse.com" after "Thu Mar 27 22:15:30 UTC 2025 - okurz@suse.com" so this package actually works as expected.

But both of that is expected. In devel:openQA openQA.changes is just used as "template" and the file _service:obs_scm:openQA.changes is the relevant one with changes. And we know that this file receives changes as we see the changelog on o3 as well as based on OSD deployments https://mailman.suse.de/mlarch/SuSE/openqa/2025/openqa.2025.04/msg00001.html

In devel:openQA:tested I assume the generated changelog is copied over as part of our submission process

mkittler wrote in #note-8:

@okurz has added the now missing changelog entries from the Factory package: https://build.opensuse.org/package/rdiff/devel:openQA/openQA?linkrev=base&rev=10701

cool, that happened automatically. I didn't do anything in particular now

I've just checked the os-autoinst package and there we have many missing changelog entries: https://build.opensuse.org/projects/devel:openQA/packages/os-autoinst/files/_service:obs_scm:os-autoinst.changes?expand=1

So this looks like the state the openQA package was in before manually adding previous log lines. A regularly checked-in _servicedata file is still present. Not sure why os-autoinst behaves differently than openQA with that file present (because openQA had a very long all-in-one changelog entry when that file was present).

It still doesn't work. Now, after new changes have been merged on master the changlog is at Update to version 5.1744638607.836ce6bd but before that there's already Update to version 5.1743174385.0bd1f0a8 and the entry Update to version 5.1744361733.ab559902 mentioned on #180068#note-8 is missing. So now we basically just have what's in the checked-in file and the most recent entry but not any older automatically generated entries. I'm not sure how to fix that. On os-autoinst it looks similar (even though I haven't touched that package at all).

For quick reference, the links to the two problematic changelog files:

• https://build.opensuse.org/projects/devel:openQA/packages/openQA/files/_service:obs_scm:openQA.changes?expand=1
• https://build.opensuse.org/projects/devel:openQA/packages/os-autoinst/files/_service:obs_scm:os-autoinst.changes?expand=1

I added the explicit _servicedata file back, setting the revision in-line with the explicit openQA.changes file. That added another changelog entry to the service-generated file which is a bit redundant but at least looks more complete.

I also asked on Slack but haven't gotten any promising answers yet.

With the explicit _servicedata file back we're back in the situation where all changes end up in one big entry.

I have checked the following as discussed in the unblock meeting:

1. Changes on https://github.com/openSUSE/obs-service-tar_scm that could have caused this regression. (I was told on Slack that this behavior (one big entry at the beginning) is expected but I still believe this is a regression.)
   1. I couldn't find a change in the relevant time-frame that would cause this.
2. How other similarly setup packages on OBS behave.
   1. The package we submit to Factory has an intact log but we manually take care of that.
   2. As already established, the os-autoinst package suffers from the same problem. I haven't modified it at all. It is currently in the state where only the few most recent changes are visible. I had the openQA package in that state as well after removing _servicedata. This file is present on the os-autoinst package, though. So I'm not sure why the os-autoinst and openQA packages behave differently but none of them behave as one would expect.
   3. The os-autoinst-scripts package also has only the few most recent changes as one single entry after the initial entry. Here we also don't have a manually checked-in _servicedata file so the behavior is consistent with how the openQA package behaved when I removed that file. The scripts repo has only < 900 commits. The fact that the changelog doesn't work there as well means that the theory that the changelog generation broke after some limit was reached is unlikely.
      1. The same is true for packages in my home repo, e.g. https://build.opensuse.org/projects/home:mkittler:vcs/packages/c++utilities/files/_service:tar_scm:c++utilities.changes?expand=1.
   4. I tried to find a package where obs_scm is used to generated changelogs just by running it server-side like in our case. However, I am not even sure where to look. I checked some random devel projects and couldn't find any relevant packages.
3. Whether these kinds of services are still supported when packaging moves to Git. If not it makes no sense to invest more time here in my opinion.
   1. As an alternative to the changelog file we could just install a file /usr/share/openqa/version.txt and display the hash from that on the footer. The link on the footer would just go to GitHub displaying the log as of this version.
   2. Answer regarding source services: Can_I_still_have_service_with_server_side_services?
      1. So these services will still be supported but with caveats.

After these findings I would conclude that this kind of source service doesn't work in the way we want. This has nothing to do with changing our versioning scheme. So I would probably change openQA to no longer use these changelogs as mentioned in point 3.1. Being independent from the concrete way we build rpm packages makes much more sense for the openQA web UI anyway.

Not sure why it previously worked better (to some extend). As mentioned on #180068#note-9 the explicitly checked in changelog was updated automatically after I removed _servicedata but then not anymore. I don't think this update had something do to with removing _servicedata, though. It happened via https://build.opensuse.org/package/rdiff/devel:openQA/openQA?linkrev=base&rev=10701 which has the commit message "sync changesrevision with openSUSE:Factory" so it is created by https://github.com/os-autoinst/scripts/blob/ca9ce69a8f7319cbb269fe4830d286fa25a41481/os-autoinst-obs-auto-submit#L140. Maybe this script doesn't handle our changed versioning scheme but then I'm wondering why the changelog still looks good in submissions to Factory.

If we decide to put more effort into it nevertheless I could setup a small dummy/test project and package to test this more efficiently as discussed in the unblock meeting. After my previous findings it would probably make more sense to debug os-autoinst-obs-auto-submit as this script is probably what caused this to work and what probably needs tweaking now.

I've just tested os-autoinst-obs-auto-submit locally to see where it might be broken:

• last_revision() still works
• The top-level submit() up to the point where sync_changesrevision() is called still works
• sync_changesrevision() itself still works, at least kind of
  • It created https://build.opensuse.org/package/rdiff/devel:openQA/openQA?linkrev=base&rev=10720 which removed the additional entry I got after adding _servicedata back.
  • It also worked before as stated in #180068#note-9.

So I'm wondering why this didn't produce any "sync changesrevision with openSUSE:Factory" changes for so long then.

This is how it would look like if we would just point to GitHub for the changelog: https://github.com/os-autoinst/openQA/pull/6396