Project

General

Profile

action #58304

A personal activity view for developers

Added by coolo over 1 year ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2019-10-17
Due date:
2020-10-17
% Done:

0%

Estimated time:
Difficulty:
Tags:

Description

Motivation

I find it very easy to get lost when doing 2 things. I.e. having tests in developer mode or scheduling jobs from git repo - or commenting on issues.

As such I would like to filter for the things I did last to pick them up again

User Story

from #60053 : I would like to have a link that lists all the tests that were started by me (or a specific user).

Acceptance criteria

  • AC1: A web page view exists showing links to most recent jobs triggered by the currently logged in user

Out of scope

  • Showing jobs or other items for any other user

Further details

  • "most recent" means that not only some jobs are shown but definitely the "most recent", i.e. highest job ids related to that user, but potentially more older jobs as well with an undefined limit. Counter-example: Not only oldest jobs or a random hit from a database search
  • The first focus can be on "jobs" though other items can be included or left for further work
  • Related: It would be nice to see the user who has started in the overview page.

Related issues

Related to openQA Project - coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settingsBlocked2018-03-232021-05-12

Related to openQA Project - action #12042: implement favorite job groupsRejected2016-05-18

Related to openQA Project - action #73567: openQA tour does not reliably save its stateResolved2020-10-19

Has duplicate openQA Project - action #60053: List all the tests I have startedRejected2019-11-19

History

#1 Updated by okurz over 1 year ago

  • Description updated (diff)

#2 Updated by okurz over 1 year ago

  • Has duplicate action #60053: List all the tests I have started added

#3 Updated by mkittler over 1 year ago

We already track job_create, job_restart and similar events by the user. So the audit log filtered by e.g. user:coolo would already show some activity but e.g. starting a developer mode session is missing. It wouldn't be hard to add missing audit events and to add a "My activity" button somewhere which would then display the filtered audit log. Of course this leaves the question which interesting audit events are missing.

It wouldn't be hurt to improve the usability of the audit log a little bit. E.g. instead of displaying { "id": "3595900", "result": { "3595900": 3597492 } } for a job_restart event we could show links to that jobs. (That could easily be done on the client-side via the DataTables API.)


That would be sufficient, right? I mean we don't want to duplicate the browser history.

#4 Updated by tinita over 1 year ago

mkittler So I guess once a job is created there is no connection to the user anymore?

#5 Updated by mkittler over 1 year ago

tinita As I said, the job_create audit event is created and that contains a reference to the user. That's why I proposed to implement this via the audit log. Otherwise we indeed needed to add a reference from the job to the user who created it.

#6 Updated by cdywan over 1 year ago

mkittler wrote:

We already track job_create, job_restart and similar events by the user. So the audit log filtered by e.g. user:coolo would already show some activity but e.g. starting a developer mode session is missing. It wouldn't be hard to add missing audit events and to add a "My activity" button somewhere which would then display the filtered audit log. Of course this leaves the question which interesting audit events are missing.

It wouldn't be hurt to improve the usability of the audit log a little bit. E.g. instead of displaying { "id": "3595900", "result": { "3595900": 3597492 } } for a job_restart event we could show links to that jobs. (That could easily be done on the client-side via the DataTables API.)

That. I brought it up before but I guess I never ended up filing the ticket for it.

I'd suggest we

  1. Collect use cases
  2. Have a renderer for each event that renders it in a user-friendly way, my first thought would be to have controllers register "event renderers" - not sure if there's an obvious mojo-esque way of doing it (helpers?)
  3. Implement that in the audit log, which is arguably barely usable as-is
  4. Consider a separate view based on the implementation

#7 Updated by mkittler over 1 year ago

cdywan

to 2.: I would do the rendering only on the client with JavaScript/DataTables. So Mojo wouldn't be involved at all.

to 4.: That would be the last point, indeed. If we decide to go that far I think this view could still share most of the code with the audit log. The controller would only filter results for the current user by default and we obviously don't want the "User" column in that view.

#8 Updated by cdywan over 1 year ago

mkittler wrote:

cdywan

to 2.: I would do the rendering only on the client with JavaScript/DataTables. So Mojo wouldn't be involved at all.

That might work. My concern would be 1) scattering the logic 2) extra calls to retrieve missing data eg. the event itself won't have all that you need, although this might not be that much of a problem. We'll need the events to be spec'ed properly in any case.

#9 Updated by okurz over 1 year ago

  • Related to coordination #58184: [saga][epic][use case] full version control awareness within openQA, e.g. user forks and branches, fully versioned test schedules and configuration settings added

#10 Updated by okurz over 1 year ago

#11 Updated by cdywan about 1 year ago

Another feedback conversation brought up the value of live updates, as you're looking at the list and expecting to see the current state when you get back to it. Notifications would even further optimize the use case of checking a finished job for the results.

openqa-mon is implementing a CLI version of that in a simple script.

#12 Updated by okurz about 1 year ago

nice idea. Would be cool to have this software available in public form.

EDIT: Talked to the author, it's public: https://github.com/grisu48/openqa-mon \o/

#13 Updated by cdywan about 1 year ago

  • Status changed from New to In Progress
  • Assignee set to cdywan
  • Target version set to Current Sprint

I've been coming back to this ticket on and off to collect user stories and actually started looking into a proof of concept for the rendering on the Javascript side, so I might as well take it.

#14 Updated by okurz about 1 year ago

I am wondering about your pick. The ticket is not properly refined, was not in "Current Sprint" and "Normal" priority. If you really fancy doing it, I can't stop you but it isn't exactly "following priority", right? :)

EDIT: Discussed with cdywan. We agreed to conduct a time boxed feasibility study with 4h to find out what is possible, how to continue, what is the expected outcome and effort.

#15 Updated by cdywan about 1 year ago

So some notes on this:

  • A look at the code revealed some inconsistencies between single and multiple event rendering, and missing CSS to make the indentation that the code already applies work: os-autoinst/openQA#2875
  • With the audit log having the basics to render events, which right now means formatting JSON, and having the ability to filter by any database column, we could add more formatters based on events types in Javascript (what Marius suggested before) and I think that seems the best option if we can re-use that code in several places
  • it would be easy to add other routes e.g. /activity with different columns, and a default filter
  • we could have a different route e.g. /group_activity which repurposes this not as a personal view but for a particular group - not the scope of this ticket strictly speaking, but this would cover more use cases and make it useful to a wider audience
  • meaningful changes can't easily be identified e.g. jobgroup_update includes all properties available, not just new values - this comes from the HTML forms, I think we'd want to fix that so that the formatter can just show what happened, by omitting values from the event that did not change
  • I was surprised to find that job_done is blacklisted. Apparently this is considered clutter. We probably want to re-consider that to get job results. De-duplication could be applied here also to make it more useful. We could also add filters for different views instead of blacklists, technically speaking this is easy to enable. We can incidentally add hidden columns and allow searching them - this was not clear to me before.

#16 Updated by cdywan about 1 year ago

  • Status changed from In Progress to New
  • Assignee deleted (cdywan)
  • Target version deleted (Current Sprint)

Resetting the status. Findings from my above time-boxed study should be evaluated to refine and possibly split up the ticket.

#17 Updated by okurz 10 months ago

  • Target version set to Ready

#18 Updated by cdywan 7 months ago

  • Assignee set to cdywan

#19 Updated by cdywan 7 months ago

  • Status changed from New to In Progress

It's worth clarifying that the fixes from #49202 made this Workable but I apparently didn't manage to save the comment here. And for the record, we don't need events for finished jobs either since we just need any event to track a given job.

https://github.com/os-autoinst/openQA/pull/3453

#20 Updated by okurz 7 months ago

cdywan wrote:

It's worth clarifying that the fixes from #49202 made this Workable

Well, I disagree with that :)

The ticket still does not have at least acceptance criteria, e.g. to fulfill https://progress.opensuse.org/projects/openqav3/wiki/#Feature-requests

I have read your comments and I think your PR is nice but I am not sure if it really "fixes" this ticket

#21 Updated by okurz 7 months ago

  • Description updated (diff)

Added user story, ACs, out of scope, etc. in description together with cdywan

#22 Updated by cdywan 7 months ago

  • Status changed from In Progress to Feedback

#23 Updated by okurz 7 months ago

As usual I think it's good to keep this ticket open until we have verification from deployment, e.g. as soon as tomorrow from o3. Any "out of scope" ideas or further ideas and feedback from users can go into #65271

#24 Updated by okurz 7 months ago

Last mentioned PR is merged and deployed on o3 and working fine, e.g. following https://openqa.opensuse.org/admin/activity_view . It takes some seconds with useful feedback that something is going on and then jobs are shown. Unfortunately does not seem to show in "most recent first". I hope this is something that can be fixed easily? I would not do any code changes on top for now.

cdywan can you / would you like to

  • add ordering by job id descending so that most recent shows up first?
  • include the menu entry in the "feature tour" or mention separately, e.g. via blog bost or just chat message?

You still have 6 days before this ticket turns 1 year old. Let's have it "Resolved" as a birthday present, shall we? :)

#25 Updated by cdywan 7 months ago

okurz wrote:

Last mentioned PR is merged and deployed on o3 and working fine, e.g. following https://openqa.opensuse.org/admin/activity_view . It takes some seconds with useful feedback that something is going on and then jobs are shown. Unfortunately does not seem to show in "most recent first". I hope this is something that can be fixed easily? I would not do any code changes on top for now.

cdywan can you / would you like to

  • add ordering by job id descending so that most recent shows up first?

That's exactly what it does. I think what you're seeing is some jobs finishing much later than others.

As mentioned in my initial research the job_done event is blocklisted which can cause this discrepancy. I'd like to suggest that we remove that.

#26 Updated by okurz 7 months ago

cdywan wrote:

okurz wrote:

Last mentioned PR is merged and deployed on o3 and working fine, e.g. following https://openqa.opensuse.org/admin/activity_view . It takes some seconds with useful feedback that something is going on and then jobs are shown. Unfortunately does not seem to show in "most recent first". I hope this is something that can be fixed easily? I would not do any code changes on top for now.

cdywan can you / would you like to

  • add ordering by job id descending so that most recent shows up first?

That's exactly what it does. I think what you're seeing is some jobs finishing much later than others.

certainly not the case. Could it be that maybe you read it out from the database ordered but put it into an unordered hash for displaying or something?

As mentioned in my initial research the job_done event is blocklisted which can cause this discrepancy. I'd like to suggest that we remove that.

I don't think this can explain what I see.

#27 Updated by cdywan 7 months ago

I proposed a PR for errors due to too many requests: https://github.com/os-autoinst/openQA/pull/3464

This only happens in more extreme cases with loads of jobs (events) owned by the same user e.g. geekotest but it's probably a good idea in general to throttle the number of requests made.

#28 Updated by cdywan 7 months ago

okurz wrote:

cdywan wrote:

okurz wrote:

Last mentioned PR is merged and deployed on o3 and working fine, e.g. following https://openqa.opensuse.org/admin/activity_view . It takes some seconds with useful feedback that something is going on and then jobs are shown. Unfortunately does not seem to show in "most recent first". I hope this is something that can be fixed easily? I would not do any code changes on top for now.

cdywan can you / would you like to

  • add ordering by job id descending so that most recent shows up first?

That's exactly what it does. I think what you're seeing is some jobs finishing much later than others.

certainly not the case. Could it be that maybe you read it out from the database ordered but put it into an unordered hash for displaying or something?

As mentioned in my initial research the job_done event is blocklisted which can cause this discrepancy. I'd like to suggest that we remove that.

I don't think this can explain what I see.

https://github.com/os-autoinst/openQA/pull/3470

#29 Updated by cdywan 7 months ago

okurz wrote:

  • include the menu entry in the "feature tour" or mention separately, e.g. via blog bost or just chat message?

https://github.com/os-autoinst/openQA/pull/3471

I plan on blogging about it on One (SUSE internal) as well.

#30 Updated by okurz 7 months ago

cdywan wrote:

okurz wrote:

  • include the menu entry in the "feature tour" or mention separately, e.g. via blog bost or just chat message?

https://github.com/os-autoinst/openQA/pull/3471

I plan on blogging about it on One (SUSE internal) as well.

this sounds good. As openQA is a true open source development you can try something that is available to the public as well. Either news.opensuse.org or twitter or facebook or mastodon or just a blog entry here on progress.opensuse.org? And then just link it to one.suse.com instead of having that the primary blog sink. However, keep in mind the one-year anniversary of this ticket coming up, so don't waste time in between ;)

#31 Updated by okurz 7 months ago

  • Due date set to 2020-10-17

#32 Updated by cdywan 7 months ago

okurz wrote:

I plan on blogging about it on One (SUSE internal) as well.
this sounds good. As openQA is a true open source development you can try something that is available to the public as well. Either news.opensuse.org or twitter or facebook or mastodon or just a blog entry here on progress.opensuse.org? And then just link it to one.suse.com instead of having that the primary blog sink. However, keep in mind the one-year anniversary of this ticket coming up, so don't waste time in between ;)

I'd like that. Unfortunately I don't have access to other channels currently but I added this to #12134 and will try and see how to do that in the future. Re-posting existing screenshots and texts should be easy enough as well.

On that note I'd like to wrap this ticket up as soon as the ordering fix is in. I'm seeing some CircleCI failures that I couldn't reproduce locally. But I got a very helpful suggestion in the meanwhile so I should have this sorted later today.

#33 Updated by cdywan 7 months ago

cdywan wrote:

On that note I'd like to wrap this ticket up as soon as the ordering fix is in. I'm seeing some CircleCI failures that I couldn't reproduce locally. But I got a very helpful suggestion in the meanwhile so I should have this sorted later today.

Unfortunately tests still don't pass on CircleCI... I changed the unit test to sort the hash keys of the fake job events, and also made sure the intentional "duplicate" event (since in real if you get multiple events for each job) but there is one remaining difference. So I'm still pondering what else could be "undefined".

#34 Updated by okurz 7 months ago

  • Related to action #73567: openQA tour does not reliably save its state added

#35 Updated by okurz 7 months ago

https://github.com/os-autoinst/openQA/pull/3470 is still open.

#73567 is a ticket about the feature tour we opened up after realizing a problem due to the work here.

#37 Updated by cdywan 6 months ago

  • Status changed from Feedback to Resolved

I'll consider this resolved from checking it on o3.

#38 Updated by okurz 6 months ago

yes, all good now :) Thanks.

Also available in: Atom PDF