QA (public) » openQA Project (public)

the job was deleted through the API while the task was running - so restart when this happens?

could this become racy when we restart tasks while multiple jobs are deleted? Would it be ok to simply wait for the next task or are subsequent tasks very cheap?

Of course this is racy and we want to avoid locking the tables - so we will have to catch the exceptions. So far the limit_assets tasks are run more frequent than necessary so just exiting the task if there is a problem is good enough.

But there is another ticket to run it on a timer base - then we'd have to restart the task, so I guess even now restarting the task is the better approach.

Another instance of the problem:

{
  "args" => [],
  "attempts" => 1,
  "children" => [],
  "created" => "2019-09-27T07:56:32.25331Z",
  "delayed" => "2019-09-27T07:56:32.25331Z",
  "finished" => "2019-09-27T07:56:35.34779Z",
  "id" => 263466,
  "notes" => {
    "gru_id" => 27377799,
    "ttl" => 172800
  },
  "parents" => [],
  "priority" => 10,
  "queue" => "default",
  "result" => {
    "error" => "\$VAR1 = bless( {\n                 'msg' => 'DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::Pg::st execute failed: ERROR:  duplicate key value violates unique constraint \"assets_type_name\"\nDETAIL:  Key (type, name)=(iso, SLE-12-SP5-SDK-DVD-x86_64-Build0268-Media1.iso) already exists. [for Statement \"INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id\" with ParamValues: 1=\\'SLE-12-SP5-SDK-DVD-x86_64-Build0268-Media1.iso\\', 2=\\'2019-09-27 07:56:32\\', 3=\\'2019-09-27 07:56:32\\', 4=\\'iso\\'] at /usr/share/openqa/script/../lib/OpenQA/Schema/ResultSet/Assets.pm line 61\n'\n               }, 'DBIx::Class::Exception' );\n",
    "output" => "[debug] [pid:6873] [DBIC] Took 0.01458800 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( type = ? ): 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-SDK-DVD-ppc64le-Build0268-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00081800 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-SDK-DVD-ppc64le-Build0268-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00137100 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-SDK-DVD-ppc64le-Build0268-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-HA-DVD-x86_64-Build0149-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.01329400 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-HA-DVD-x86_64-Build0149-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00062100 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-HA-DVD-x86_64-Build0149-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-Server-DVD-ppc64le-Build0339-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00041800 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-Server-DVD-ppc64le-Build0339-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00046400 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-Server-DVD-ppc64le-Build0339-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-HA-GEO-DVD-s390x-x86_64-Build0119-Media.iso\n[debug] [pid:6873] [DBIC] Took 0.00055100 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-HA-GEO-DVD-s390x-x86_64-Build0119-Media.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00038800 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-HA-GEO-DVD-s390x-x86_64-Build0119-Media.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-WE-DVD-x86_64-Build0162-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00034900 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-WE-DVD-x86_64-Build0162-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00028300 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-WE-DVD-x86_64-Build0162-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-Server-DVD-aarch64-Build0339-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00034000 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-Server-DVD-aarch64-Build0339-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00031300 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-Server-DVD-aarch64-Build0339-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-SDK-DVD-aarch64-Build0268-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00037900 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-SDK-DVD-aarch64-Build0268-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00028700 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-SDK-DVD-aarch64-Build0268-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-SAP-DVD-x86_64-Build0312-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00022400 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-SAP-DVD-x86_64-Build0312-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00022500 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-SAP-DVD-x86_64-Build0312-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-SAP-DVD-ppc64le-Build0312-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00036600 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-SAP-DVD-ppc64le-Build0312-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00030600 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-SAP-DVD-ppc64le-Build0312-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-HA-DVD-ppc64le-Build0149-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00032300 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-HA-DVD-ppc64le-Build0149-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00030200 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-HA-DVD-ppc64le-Build0149-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-Server-MINI-ISO-x86_64-Build0339-Media.iso\n[debug] [pid:6873] [DBIC] Took 0.00027700 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-Server-MINI-ISO-x86_64-Build0339-Media.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00035600 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-Server-MINI-ISO-x86_64-Build0339-Media.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-Server-DVD-x86_64-Build0339-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00026300 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-Server-DVD-x86_64-Build0339-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00024200 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-Server-DVD-x86_64-Build0339-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-HA-DVD-s390x-Build0149-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00023400 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-HA-DVD-s390x-Build0149-Media1.iso', 'iso'\n[debug] [pid:6873] [DBIC] Took 0.00021400 seconds: INSERT INTO assets ( name, t_created, t_updated, type) VALUES ( ?, ?, ?, ? ) RETURNING id: 'SLE-12-SP5-HA-DVD-s390x-Build0149-Media1.iso', '2019-09-27 07:56:32', '2019-09-27 07:56:32', 'iso'\n[info] [pid:6873] Registering asset iso/SLE-12-SP5-SDK-DVD-x86_64-Build0268-Media1.iso\n[debug] [pid:6873] [DBIC] Took 0.00024200 seconds: SELECT me.id, me.type, me.name, me.size, me.checksum, me.last_use_job_id, me.fixed, me.t_created, me.t_updated FROM assets me WHERE ( ( me.name = ? AND me.type = ? ) ): 'SLE-12-SP5-SDK-DVD-x86_64-Build0268-Media1.iso', 'iso'\n"
  },
  "retried" => undef,
  "retries" => 0,
  "started" => "2019-09-27T07:56:32.2584Z",
  "state" => "failed",
  "task" => "limit_assets",
  "time" => "2019-09-30T09:38:33.61567Z",
  "worker" => 198
}

This time it is a unique key constraint. I guess for the sake of the ticket is doesn't matter whether it is a unique for foreign key constraint. So I'm implementing a retry in any case.

PR: https://github.com/os-autoinst/openQA/pull/2360