action #46802

Replace D-Bus with plain HTTP

Added by kraih about 1 year ago. Updated 9 months ago.

Status:ResolvedStart date:29/01/2019
Priority:NormalDue date:
Assignee:kraih% Done:

0%

Category:Feature requests
Target version:Current Sprint
Difficulty:
Duration:

Description

In preparation of replacing dbus with redis we first need to find the right way to add
redis.

  • Choose the right Redis module to use (most likely Mojo::Redis)
  • Package the module so our containers can use it
  • Add a message passing module that provides a similar abstraction layer to how we use dbus currently
  • Provide a simple way to start a redis server for tests (just like Postgres)

Related issues

Related to openQA Project - action #46778: [epic] Replace dbus with HTTP Resolved 28/01/2019
Blocks openQA Project - action #47117: EPIC: Fix worker->websocket->scheduler->webui connection New 04/02/2019

History

#1 Updated by kraih about 1 year ago

#2 Updated by kraih about 1 year ago

#3 Updated by kraih about 1 year ago

#4 Updated by kraih about 1 year ago

  • Related to deleted (action #46778: [epic] Replace dbus with HTTP)

#5 Updated by kraih about 1 year ago

#6 Updated by coolo about 1 year ago

  • Blocks action #47117: EPIC: Fix worker->websocket->scheduler->webui connection added

#7 Updated by kraih 11 months ago

After it turned out that the resource allocator could just be removed without adding a replacement, i started to wonder if that might be the case for all our current uses of dbus. While it's not quite as straight forward for the websocket server and scheduler, there might be better options than using redis. I'm currently exploring the use of postgres pub/sub and/or plain HTTP as a replacement for dbus rpc. This might take a little time to figure out due to my limited understanding of the big picture, but the much simpler overall architecture should be worth it.

#8 Updated by kraih 11 months ago

  • Status changed from New to In Progress

#9 Updated by kraih 11 months ago

Also worth mentioning that my original proposal to use redis was based on the assumption that dbus was used for message passing in openQA. But that turned out to be false, in all instances it is used exclusively for blocking RPC.

#10 Updated by kraih 11 months ago

The websocket server has been refactored into a well structured Mojolicious application now. So everything is prepared for using HTTP as a new RPC mechanism, which will be the next step. https://github.com/os-autoinst/openQA/pull/2037

#11 Updated by mkittler 11 months ago

  • Subject changed from Add basic redis support to Replace D-Bus with plain HTTP

#12 Updated by kraih 11 months ago

I've opened a PR https://github.com/os-autoinst/openQA/pull/2043 for replacing D-Bus with HTTP in the websocket server. Now it just needs to be tested on the staging machines.

#13 Updated by kraih 9 months ago

Opened another (and hopefully final) PR for removing the final traces of D-Bus in openQA. https://github.com/os-autoinst/openQA/pull/2090

#14 Updated by okurz 9 months ago

  • Category set to Feature requests

merged, also the update of the architecture diagram.

#15 Updated by okurz 9 months ago

deployed on o3 since yesterday without any visible problems. I wonder about one thing: Do we still have AMQP messages output? Not sure if this is related but the only thing I could think of right now that can still go wrong ;)

#16 Updated by coolo 9 months ago

how is amqp relate to dbus and/or the scheduler?

#17 Updated by okurz 9 months ago

hopefully not at all :) Just thought maybe some "events" would be hijacked which are not there even more. But if you say it's not the case I am sure we are good.

#18 Updated by kraih 9 months ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF