QA » openQA Project

I did a test using openqa_review to get comments from https://bugzilla.suse.com/show_bug.cgi?id=1005824. it only can get the public comments. I tried this use my own account, all comments I could get. So I think openqa_review does not have the permission. After I changed the is_private to False, the comment was added successfully. See: https://bugzilla.suse.com/show_bug.cgi?id=1005824 comments 11 and 13.
Not sure if we should upgrade openqa_review permission, I created a pr to change the parameter: https://github.com/os-autoinst/openqa_review/pull/139. If we need to update the permission, I can close the pull request.

I run openqa-review locally
python2.7 -m openqa_review.openqa_review --host http://openqa.suse.de -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 --job-groups '15 SP3' --reminder-comment-on-issues

After changing the is_private : False, the comment was added to bugzilla successfully.
see: https://bugzilla.suse.com/show_bug.cgi?id=1179434 comment#27

But CI pipeline still report error 500: 500 Server Error: Internal Server Error for url : https://gitlab.suse.de/openqa/openqa-review/-/jobs/349401/raw

I tested in container:
liuxiaojing@linux-90dc:~> sudo podman run --rm -it registry.suse.de/home/okurz/container/images/openqa-review /bin/bash
According to gitlab log message:

$ sh -ex bin/openqa-review-daily-email $ARGS
++ echo '$ sh -ex bin/openqa-review-daily-email $ARGS'
++ sh -ex bin/openqa-review-daily-email
++ dirname bin/openqa-review-daily-email
+ . bin/_common
++ set -o pipefail
+ '[' -z ']'
+ setup_tmpdir
++ mktemp -d
+ tmp=/tmp/tmp.uXWMAvvkIL
+ trap cleanup EXIT
+ openqa_host=https://openqa.suse.de
+ recv=openqa-suse-status@suse.de
+ html_target_file=openqa_suse_de_status.html
+ html_target=openqa_suse_de_status.html
+ openqa_review_args='--host https://openqa.suse.de -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 --exclude-job-groups ^(Released|Development|old)'
+ load_args='--load --load-dir=/tmp/tmp.uXWMAvvkIL'
+ openqa_review_email_args='--load --load-dir=/tmp/tmp.uXWMAvvkIL'
+ openqa_review_html_args='--load --load-dir=/tmp/tmp.uXWMAvvkIL'
+ openqa_review_save_args='--reminder-comment-on-issues --save --save-dir /tmp/tmp.uXWMAvvkIL'
++ command -v openqa-review
+ openqa_review=/usr/bin/openqa-review
++ dirname bin/openqa-review-daily-email
+ TPL=bin/../dashboard_files/dashboard.html.in
++ /usr/bin/openqa-review --host https://openqa.suse.de -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 --exclude-job-groups '^(Released|Development|old)' --reminder-comment-on-issues --save --save-dir /tmp/tmp.uXWMAvvkIL

It runs /usr/bin/openqa-review ..., this file belongs:

32c2adb65251:/openqa_review # rpm -qf /usr/bin/openqa-review
python38-openqa_review-1.20.2-1.3.noarch

this package doesn't include the pr https://github.com/os-autoinst/openqa_review/pull/139.
Not sure why we do git clone https://github.com/os-autoinst/openqa_review.git . in gitlab, we don't use these codes.
Should we update the container image to let it includes the change (https://github.com/os-autoinst/openqa_review/pull/139)? or change the gitlab-ci to use the github code?

Xiaojing_liu wrote:

I tested in container:
liuxiaojing@linux-90dc:~> sudo podman run --rm -it registry.suse.de/home/okurz/container/images/openqa-review /bin/bash
According to gitlab log message:

$ sh -ex bin/openqa-review-daily-email $ARGS
++ echo '$ sh -ex bin/openqa-review-daily-email $ARGS'
++ sh -ex bin/openqa-review-daily-email
++ dirname bin/openqa-review-daily-email
+ . bin/_common
++ set -o pipefail
+ '[' -z ']'
+ setup_tmpdir
++ mktemp -d
+ tmp=/tmp/tmp.uXWMAvvkIL
+ trap cleanup EXIT
+ openqa_host=https://openqa.suse.de
+ recv=openqa-suse-status@suse.de
+ html_target_file=openqa_suse_de_status.html
+ html_target=openqa_suse_de_status.html
+ openqa_review_args='--host https://openqa.suse.de -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 --exclude-job-groups ^(Released|Development|old)'
+ load_args='--load --load-dir=/tmp/tmp.uXWMAvvkIL'
+ openqa_review_email_args='--load --load-dir=/tmp/tmp.uXWMAvvkIL'
+ openqa_review_html_args='--load --load-dir=/tmp/tmp.uXWMAvvkIL'
+ openqa_review_save_args='--reminder-comment-on-issues --save --save-dir /tmp/tmp.uXWMAvvkIL'
++ command -v openqa-review
+ openqa_review=/usr/bin/openqa-review
++ dirname bin/openqa-review-daily-email
+ TPL=bin/../dashboard_files/dashboard.html.in
++ /usr/bin/openqa-review --host https://openqa.suse.de -n -r -T --query-issue-status --no-empty-sections --include-softfails --running-threshold=2 --exclude-job-groups '^(Released|Development|old)' --reminder-comment-on-issues --save --save-dir /tmp/tmp.uXWMAvvkIL

It runs /usr/bin/openqa-review ..., this file belongs:

32c2adb65251:/openqa_review # rpm -qf /usr/bin/openqa-review
python38-openqa_review-1.20.2-1.3.noarch

this package doesn't include the pr https://github.com/os-autoinst/openqa_review/pull/139.
Not sure why we do git clone https://github.com/os-autoinst/openqa_review.git . in gitlab, we don't use these codes.
Should we update the container image to let it includes the change (https://github.com/os-autoinst/openqa_review/pull/139)? or change the gitlab-ci to use the github code?

We do clone the repo in GitLab from what I see here https://gitlab.suse.de/openqa/openqa-review/-/blob/master/.gitlab-ci.yml#L37 - or did you mean something else?

But I wonder, why don't we make sure to update the image and use it everywhere?

We do clone the repo in GitLab from what I see here https://gitlab.suse.de/openqa/openqa-review/-/blob/master/.gitlab-ci.yml#L37 - or did you mean something else?

But I wonder, why don't we make sure to update the image and use it everywhere?

Yes, we clone the repo in GitLab. But I guess we don't use the github code to do openqa-review.
According to log message, it runs /usr/bin/openqa-review, and we could see

32c2adb65251:/openqa_review # rpm -qf /usr/bin/openqa-review
python38-openqa_review-1.20.2-1.3.noarch
32c2adb65251:/openqa_review # rpm -ql python38-openqa_review-1.20.2-1.3.noarch
...
/usr/bin/openqa-review
...
/usr/lib/python3.8/site-packages/openqa_review/browser.py
/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py

So the github code isn't run, right? If I understand correctly, I wonder, why we clone github code here, we don't use it.

The latest container image is created on 29, Jan 2021 : https://registry.suse.de/cgi-bin/cooverview?srch_term=project%3D%5Ehome%3Aokurz#layers-17711ab9a6ab

Xiaojing_liu wrote:

We do clone the repo in GitLab from what I see here https://gitlab.suse.de/openqa/openqa-review/-/blob/master/.gitlab-ci.yml#L37 - or did you mean something else?

But I wonder, why don't we make sure to update the image and use it everywhere?

Yes, we clone the repo in GitLab. But I guess we don't use the github code to do openqa-review.
According to log message, it runs /usr/bin/openqa-review, and we could see

32c2adb65251:/openqa_review # rpm -qf /usr/bin/openqa-review
python38-openqa_review-1.20.2-1.3.noarch
32c2adb65251:/openqa_review # rpm -ql python38-openqa_review-1.20.2-1.3.noarch
...
/usr/bin/openqa-review
...
/usr/lib/python3.8/site-packages/openqa_review/browser.py
/usr/lib/python3.8/site-packages/openqa_review/openqa_review.py

So the github code isn't run, right? If I understand correctly, I wonder, why we clone github code here, we don't use it.

Upon closer inspection I see that bin/openqa-review-daily-email uses the repo, but said shellscript calls command -v openqa-review unless $openqa_review overrides it.

I'd vote to fix that default to call ./openqa_review/openqa_review.py. You might be wrapping up your day by now... so I can offer proposing a branch to fix that.

cdywan wrote:

Upon closer inspection I see that bin/openqa-review-daily-email uses the repo, but said shellscript calls command -v openqa-review unless $openqa_review overrides it.

I'd vote to fix that default to call ./openqa_review/openqa_review.py. You might be wrapping up your day by now... so I can offer proposing a branch to fix that.

https://github.com/os-autoinst/openqa_review/pull/141

The problem is that I forgot to create a new package release for python-openqa_review which I can do the next days. As alternative you might be able to create a submission in OBS as well but I think we need a tag in the git repo first as well.

So what I did, in a checkout of openqa_review:

git fetch origin
git co -b fix/private_comments origin/master
git tag -f 1.20.4 && git push -f origin 1.20.4 && git push -f origin HEAD:master

then in a checkout of https://build.opensuse.org/package/show/devel:languages:python/python-openqa_review

osc up && ./_update.sh

which created
https://build.opensuse.org/request/show/876941
I suggest we wait for that and see if a new image gets built. After that we check the new run of CI pipeline and see if the fix is effective. Another story what we identified is that it can appear inconsistent to use parts of scripts from a git checkout and parts from an installed binary as well. As using the package allows better use of our own products, read "eat your own dogfood", we think it's a good idea to keep using the package but then streamline the pipeline to not use a git checkout but rely on the package completely or switch to git-only -> #89536

Thanks. Awesome achievement!