Project

General

Profile

Actions

action #133232

closed

o3 hook scripts are triggered but no comment shows up on job

Added by okurz over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
Regressions/Crashes
Target version:
Start date:
2023-07-24
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://openqa.opensuse.org/minion/jobs?state=finished&offset=0&task=hook_script shows minion jobs like https://openqa.opensuse.org/minion/jobs?id=2734386 which say

args:
- env from_email=o3-admins@suse.de scheme=http enable_force_result=true email_unreviewed=true
  exclude_group_regex='(Development|Open Build Service|Others|Kernel).*/.*' /opt/os-autoinst-scripts/openqa-label-known-issues-and-investigate-hook
- 3449226
- delay: 60
  kill_timeout: 30s
  retries: 1440
  skip_rc: 142
  timeout: 5m
attempts: 1
children: []
created: 2023-07-24T17:44:04.227673Z
delayed: 2023-07-24T17:45:08.865413Z
expires: ~
finished: 2023-07-24T17:45:23.297123Z
id: 2734386
lax: 0
notes:
  hook_cmd: env from_email=o3-admins@suse.de scheme=http enable_force_result=true
    email_unreviewed=true exclude_group_regex='(Development|Open Build Service|Others|Kernel).*/.*'
    /opt/os-autoinst-scripts/openqa-label-known-issues-and-investigate-hook
  hook_rc: 5
  hook_result: ''
parents: []
priority: 0
queue: default
result: ~
retried: 2023-07-24T17:44:08.865413Z
retries: 1
started: 2023-07-24T17:45:13.232022Z
state: finished
task: hook_script
time: 2023-07-24T17:53:29.538536Z
worker: 1690

but no comment ever shows up on https://openqa.opensuse.org/tests/3449226

Expected result

I would assume a comment with results from the above script show up on https://openqa.opensuse.org/tests/3449226#comments

Logs

gru journal related to that specific job:

Jul 24 17:45:18 new-ariel openqa-gru[7556]: jq (152 /opt/os-autoinst-scripts/openqa-investigate): jq: error (at <stdin>:1): Cannot index array with string "id" (rc: 5 Input: >>>[]<<<)

line 152 /opt/os-autoinst-scripts/openqa-investigate:

comment_id=$("${client_call[@]}" -X POST jobs/"$first_cluster_job_id"/comments text="Starting investigation for job $id" | runjq -r '.id') || return $?

access.log:

10.150.1.11 - - [24/Jul/2023:17:45:18 +0000] "POST /api/v1/jobs/3449225/comments HTTP/1.1" 301 169 "-" "openqa-investigate (https://github.com/os-autoinst/scripts)"
10.150.1.11 - - [24/Jul/2023:17:45:18 +0000] "GET /api/v1/jobs/3449225/comments HTTP/1.1" 200 2 "-" "openqa-investigate (https://github.com/os-autoinst/scripts)"

Related issues 1 (0 open1 closed)

Related to openQA Infrastructure - action #132143: Migration of o3 VM to PRG2 - 2023-07-19 size:MResolvednicksinger2023-06-29

Actions
Actions #1

Updated by okurz over 1 year ago

  • Related to action #132143: Migration of o3 VM to PRG2 - 2023-07-19 size:M added
Actions #2

Updated by tinita over 1 year ago

  • Description updated (diff)
Actions #3

Updated by tinita over 1 year ago

for some reason talking to http vs. https does something completely different:

geekotest@new-ariel:~> openqa-cli api --host http://openqa.opensuse.org -X POST jobs/3449226/comments text="test comment functionality"
[]
geekotest@new-ariel:~> openqa-cli api --host https://openqa.opensuse.org -X POST jobs/3449226/comments text="test comment functionality"
{"id":453608}
Actions #4

Updated by favogt over 1 year ago

tinita wrote:

for some reason talking to http vs. https does something completely different:

geekotest@new-ariel:~> openqa-cli api --host http://openqa.opensuse.org -X POST jobs/3449226/comments text="test comment functionality"
[]
geekotest@new-ariel:~> openqa-cli api --host https://openqa.opensuse.org -X POST jobs/3449226/comments text="test comment functionality"
{"id":453608}

This sounds like the redirect bug I encountered: The http:// -> https:// redirect makes mojo forget the method, so it ends up doing a GET request.

You can confirm that with MOJO_CLIENT_DEBUG=1 openqa-cli api or just doing openqa-cli api --host http://openqa.opensuse.org -X POST jobs/3449226/comments text="test comment functionality" now that there is a comment.

Actions #5

Updated by tinita over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to tinita
Actions #6

Updated by tinita over 1 year ago

Ok, so we (Fabian, Nick, Oli and me) solved that in a jitsi meeting.
We are now doing a 308 redirect instead of a 301. 308 also works for POST requests.
hook scripts are working again now (gru journal looks fine), waiting for investigate jobs showing up.

Actions #7

Updated by tinita over 1 year ago

There is another problem. Our hook scripts use curl to fetch data, and we're getting a redirect now:

curl http://openqa.opensuse.org/tests/3451199/file/os-autoinst.log.txt
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx/1.21.5</center>
</body>
</html>

I guess we just need to add a -L to every curl command.

Actions #8

Updated by tinita over 1 year ago

Actions #10

Updated by tinita over 1 year ago

  • Status changed from In Progress to Resolved

I can see that investigate jobs are created again and comments written.

Actions

Also available in: Atom PDF