action #124739
Updated by okurz almost 2 years ago
## Observation Dominique Leuenberger brought up in https://suse.slack.com/archives/C02CANHLANP/p1676644670756539 > (Dominique Leuenberger) openQA going down on a friday afternoon/evening? Hope that was not a deployment gone wrong > (Dominique Leuenberger sadly - that seems exactly what's happening: 2023-02-17 14:33:21|command|root@ariel|'zypper' '-n' '--no-refresh' 'dup' '--auto-agree-with-licenses' '--download-in-advance'| > (Oliver Kurz) let me check that. openqa-webui-daemon[16374]: run(/usr/bin/ruby) failed: Permission denied at /usr/lib/perl5/vendor_perl/5.26.1/Mojolicious/Plugin/AssetPack/Pipe.pm line 27. > (Oliver Kurz) https://openqa.opensuse.org/ up again. Will report a ticket and handle in there. > (Oliver Kurz) by the way we have continuous deployment of openQA+os-autoinst so this was triggered automatically. I guess it was an acceptance of some AssetPacke related package that triggered a build chain and eventually the deployment > (Sebastian Riedel) Sass dependency issue? > (Oliver Kurz) two issues actually, apparmor preventing to run sass and sass was not installed. I logged into o3 and found the openqa-webui service failed with the error as above about permission denied. So I called ``` aa-complain /etc/apparmor.d/usr.share.openqa.script.openqa && systemctl start openqa-webui; journalctl -f -u openqa-webui ``` which showed ``` Feb 17 14:49:59 ariel systemd[1]: Started The openQA web UI. … Feb 17 14:50:02 ariel openqa-webui-daemon[26137]: run(/bin/sass) failed: No such file or directory at /usr/lib/perl5/vendor_perl/5.26.1/Mojolicious/Plugin/AssetPack/Pipe.pm line 27. ``` then I did ``` zypper -n in ruby2.5-rubygem-sass && systemctl restart openqa-webui; journalctl -f -u openqa-webui ``` that worked. So two issues actually, apparmor preventing to run sass and sass was not installed. I switched back to aa-enforce, restarted and again the permission denied problem. ## Acceptance criteria * **AC1:** Original issue has been identified and mitigated for all users, not just o3 * **AC2:** o3 runs the current openQA webUI version with apparmor enforced ## Suggestions * Identify what caused the actual issue * Fix the issue for all users, not just the o3 case -> for now solved with revert https://github.com/os-autoinst/openQA/pull/5011 * ~~Add Add "sass" to apparmor files~~ files * ~~Ensure Ensure sass is pulled in as dependency where necessary~~ necessary * Run `aa-enforce /etc/apparmor.d/usr.share.openqa.script.openqa` again and ensure the webUI to be properly running * Remove sass from o3 again * Wait for actual fix in #122440 and check inside those OBS checks for the error * Ensure that the script generate-packed-assets actually fails on error and OBS check should also fail! * Cleanup not needed temporary OBS repos