QA » openQA Project » openQA Tests

We need to clarify what does it means "explicitly reproduce" new test suite? improve existing system_performance test suite?

Probably can be split.

Not enough capacity during sprint 27. And this is better done after: #41462

#41462 closed, now we can try again. Discussed in SP

Actually working on this for a while, I wonder whats the difference to 'snapper_cleanup'?

Since I failed to deliver within a week I will unassign.

@szarate I don't think your change of the subject is correct. We already explicitly trigger btrfs maintenance, see the test module "force_schedule_tasks". This ticket here is about not triggering maintenance tasks explicitly.

I am unassigning me, I did not make enough progress to believe I can create a good reproducer now. As promised, here is a summary of what I did and what I think should be done next:

My approach¶

Create dummy data in a subvolume, then create a btrfs snapshot of it
(a new subvolume that shares data with the snapshotted one). Delete
data in the original subvolume, so only the snapshot is "pointing to
it". The idea here is that btrfs metadata is created and changed,
causing btrfs balance some work.

Read out /proc/diskstats for IO queue to determine load and later give
stress-ng a try by running a benchmark at the beginning and at the
end, while btrfs maintenance jobs do their work.

I ran the tests on Tumbleweed while developing. While it is different
from SLE12 in regards to kernel / filesystem, the very same bug hit me
two weeks ago on my TW Laptop, so it is not (completely) fixed in TW
yet.

Why didn't I reproduce bsc#1063638?¶

Firstly, my approach of creating work for btrfs to clean up was probably
not good enough. I tried writing data in different ways, but
ultimately it was not enough to cause changes in system performance
when running the maintenance jobs.

Secondly, one issue I had was starting different things that cause the
load asynchronously in openQA. There might be a simple way to solve
this part, but I did not come across it.

Future¶

There are two things that immediately come to my mind for future work on this ticket:

• There were a few comments on my WIP PR (https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6487), those could be included in the test.
• Find a way to create more "dirt" for btrfs to clean up during its maintenance task

I'm taking this one, let's see if a combination of dd and bonie++ can do something

I have used Alex's attempt and modified it a bit to use /dev/urandom instead and also modified the size of the files to generate a bit more load.

I managed to get from time to time btrfs balance to stall the system for a moment, but nothing conclusive. I'm unassigning for the time being, rather than waste more time. Perhaps a look to how btrfs scrub works and some white box testing could year better results.

I also suggest to lower the priority of the ticket.

Downgrading to normal for the time being.

Bug resolved:fixed
https://bugzilla.opensuse.org/show_bug.cgi?id=1063638