Project

General

Profile

action #46802

Replace D-Bus with plain HTTP

Added by kraih over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2019-01-29
Due date:
% Done:

0%

Estimated time:
Difficulty:
Duration:

Description

In preparation of replacing dbus with redis we first need to find the right way to add
redis.

  • Choose the right Redis module to use (most likely Mojo::Redis)
  • Package the module so our containers can use it
  • Add a message passing module that provides a similar abstraction layer to how we use dbus currently
  • Provide a simple way to start a redis server for tests (just like Postgres)

Related issues

Related to openQA Project - action #46778: [epic] Replace dbus with HTTPResolved2019-01-28

Blocks openQA Project - action #47117: [epic] Fix worker->websocket->scheduler->webui connectionResolved2019-02-04

History

#1 Updated by kraih over 1 year ago

#2 Updated by kraih over 1 year ago

#3 Updated by kraih over 1 year ago

#4 Updated by kraih over 1 year ago

  • Related to deleted (action #46778: [epic] Replace dbus with HTTP)

#5 Updated by kraih over 1 year ago

#6 Updated by coolo over 1 year ago

  • Blocks action #47117: [epic] Fix worker->websocket->scheduler->webui connection added

#7 Updated by kraih over 1 year ago

After it turned out that the resource allocator could just be removed without adding a replacement, i started to wonder if that might be the case for all our current uses of dbus. While it's not quite as straight forward for the websocket server and scheduler, there might be better options than using redis. I'm currently exploring the use of postgres pub/sub and/or plain HTTP as a replacement for dbus rpc. This might take a little time to figure out due to my limited understanding of the big picture, but the much simpler overall architecture should be worth it.

#8 Updated by kraih over 1 year ago

  • Status changed from New to In Progress

#9 Updated by kraih over 1 year ago

Also worth mentioning that my original proposal to use redis was based on the assumption that dbus was used for message passing in openQA. But that turned out to be false, in all instances it is used exclusively for blocking RPC.

#10 Updated by kraih over 1 year ago

The websocket server has been refactored into a well structured Mojolicious application now. So everything is prepared for using HTTP as a new RPC mechanism, which will be the next step. https://github.com/os-autoinst/openQA/pull/2037

#11 Updated by mkittler over 1 year ago

  • Subject changed from Add basic redis support to Replace D-Bus with plain HTTP

#12 Updated by kraih about 1 year ago

I've opened a PR https://github.com/os-autoinst/openQA/pull/2043 for replacing D-Bus with HTTP in the websocket server. Now it just needs to be tested on the staging machines.

#13 Updated by kraih about 1 year ago

Opened another (and hopefully final) PR for removing the final traces of D-Bus in openQA. https://github.com/os-autoinst/openQA/pull/2090

#14 Updated by okurz about 1 year ago

  • Category set to Feature requests

merged, also the update of the architecture diagram.

#15 Updated by okurz about 1 year ago

deployed on o3 since yesterday without any visible problems. I wonder about one thing: Do we still have AMQP messages output? Not sure if this is related but the only thing I could think of right now that can still go wrong ;)

#16 Updated by coolo about 1 year ago

how is amqp relate to dbus and/or the scheduler?

#17 Updated by okurz about 1 year ago

hopefully not at all :) Just thought maybe some "events" would be hijacked which are not there even more. But if you say it's not the case I am sure we are good.

#18 Updated by kraih about 1 year ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF