https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842021-03-12T19:49:44ZopenSUSE Project Management ToolQA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3912712021-03-12T19:49:44Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>I just did a little fix to the code reading Makefile.PL to also fetch the TEST_REQUIRES.</p>
<p>We should still add META.json, and additionally check the <code>dynamic_config</code> property - if false, we don't need to read Makefile.PL, Build.PL etc.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3912742021-03-12T19:50:35Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Project</strong> changed from <i>QA</i> to <i>openQA Project</i></li><li><strong>Category</strong> set to <i>Regressions/Crashes</i></li></ul> QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3912772021-03-12T19:51:58Zokurzokurz@suse.com
<ul></ul><p>I think the generic "QA" project is better fitting. Or is it really connected to "openQA"?</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3912802021-03-12T20:10:05Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Project</strong> changed from <i>openQA Project</i> to <i>QA</i></li><li><strong>Category</strong> deleted (<del><i>Regressions/Crashes</i></del>)</li></ul><p>Ok</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3923452021-03-17T14:48:47Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>merged <a href="https://github.com/openSUSE/cpanspec/pull/28" class="external">https://github.com/openSUSE/cpanspec/pull/28</a></p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3923512021-03-17T15:03:23Zokurzokurz@suse.com
<ul></ul><p>ok, what's left to call this ticket "Resolved" after the merged PR?</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3923662021-03-17T15:04:05Zokurzokurz@suse.com
<ul><li><strong>Due date</strong> set to <i>2021-03-31</i></li></ul><p>Setting due date based on mean cycle time of SUSE QE Tools</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3923752021-03-17T15:05:18Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>What's left? Reading META.json and also decide further actions depending on dynamic_config.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3955582021-04-07T16:29:51Zlivdywanliv.dywan@suse.com
<ul><li><strong>Due date</strong> changed from <i>2021-03-31</i> to <i>2021-04-16</i></li></ul><p>Bumping <em>due date</em> due to vacation. </p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3992282021-04-20T16:06:41Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>I ran the current cpanspec on about 20% of the modules (from perl-ACH to perl-Data-Munge) in devel:languages:perl, looked if the changes are expected and committed them to a local git repository.</p>
<p>Now I can make changes to cpanspec and run it on all of those modules to ensure there are no unexpected changes.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3992312021-04-20T16:22:33Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>I'm also creating a Dockerfile, so in the future it will be easier to install the cronjob on a new server and get all the dependencies and environment variables right.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3993572021-04-21T09:01:39Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Due date</strong> changed from <i>2021-04-16</i> to <i>2021-04-23</i></li></ul> QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3995852021-04-21T18:31:08Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>I made progress with the dependency handling.</p>
<p>Now the script is reading META.json first.<br>
If META.json is missing or didn't contain any prereqs, META.yml is read and checked for prereqs.</p>
<p>If prereqs have been identified and <code>dynamic_config</code> is false, the step to execute Makefile.PL is skipped.</p>
<p>Also refactored a bit and moved things into subroutines.</p>
<p>Still running the last tests but it looks good.</p>
<p>The next step would be to check the <code>license</code> field, and maybe improve the <code>abstract</code> handling (<code>abstract</code> is a required field in <code>META.json</code>, so we can skip reading it from POD in this case).</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=3999722021-04-22T18:55:29Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>I ran some more tests and fixed some minor bugs, and also stumbled over a bug related to <code>@INC</code>.<br>
Data::Rand::Obscure has this Build.PL:<br>
<a href="https://metacpan.org/source/RKRIMEN/Data-Rand-Obscure-0.021/Build.PL" class="external">https://metacpan.org/source/RKRIMEN/Data-Rand-Obscure-0.021/Build.PL</a></p>
<p>Because the <code>.</code> has been removed from <code>@INC</code>, what happened with the current cpanspec version is that cpanspec's root path was added to <code>@INC</code>, so it loaded cpanspec's own Makefile.PL instead of the module's one, which lead to the following Require entries <a href="https://build.opensuse.org/package/view_file/devel:languages:perl/perl-Data-Rand-Obscure/perl-Data-Rand-Obscure.spec?expand=1" class="external">https://build.opensuse.org/package/view_file/devel:languages:perl/perl-Data-Rand-Obscure/perl-Data-Rand-Obscure.spec?expand=1</a>:</p>
<pre><code>BuildRequires: perl-macros
BuildRequires: perl(Archive::Tar)
BuildRequires: perl(Archive::Zip)
BuildRequires: perl(Carp::Clan)
BuildRequires: perl(IO::Uncompress::Bunzip2)
BuildRequires: perl(LWP::UserAgent)
BuildRequires: perl(Module::Build)
BuildRequires: perl(Object::Tiny)
BuildRequires: perl(Parse::CPAN::Packages)
BuildRequires: perl(Pod::Simple::TextContent)
BuildRequires: perl(Text::Autoformat)
BuildRequires: perl(YAML)
Requires: perl(Archive::Tar)
Requires: perl(Archive::Zip)
Requires: perl(Carp::Clan)
Requires: perl(IO::Uncompress::Bunzip2)
Requires: perl(LWP::UserAgent)
Requires: perl(Object::Tiny)
Requires: perl(Parse::CPAN::Packages)
Requires: perl(Pod::Simple::TextContent)
Requires: perl(Text::Autoformat)
Requires: perl(YAML)
</code></pre>
<p>My new version moved the code around and didn't put the cpanspec root path to <code>@INC</code> anymore, so I stumbled upon this.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=4002862021-04-23T21:54:44Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>I cleaned up the new code a bit, added a debug mode.<br>
Also I added parseable statistics output (YAML :). With this we can parse the logs of the automatic CPAN-* and d:l:p updates.<br>
This will give us easy access to the meta data of the newest releases.</p>
<p>The new code is also a bit faster (if running Makefile.PL/etc. and/or PPI parsing can be skipped, which is more likely the case for newer modules/releases).<br>
A quick benchmark for 50 modules in d:l:p: 12m5.787s (old) vs. 9m20.203s (new).</p>
<p>I will create an extra ticket for the license improvements, and the statistic data that has been collected until then can be used.</p>
<p>I will run <a href="https://github.com/perlpunk/cpanspec/tree/metajson" class="external">this branch</a> on the autoupdate host for a few days and monitor, and if it looks fine, make a PR.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=4013492021-04-27T15:49:54Ztinitatina.mueller+trick-redmine@suse.com
<ul></ul><p>I rolled out the code on the autoupdate host and will monitor for the next few days.<br>
I installed logrotate for the logfiles (which is something that I wanted to do a long time ago), so logs from the previous day aren't overwritten anymore.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=4013612021-04-27T16:39:02Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Due date</strong> changed from <i>2021-04-23</i> to <i>2021-04-30</i></li></ul><p>Moving due date.</p>
<p>Running the current code on a significant amount of packages and comparing results of each package to filter out existing problems or packages that should be left alone took a bit longer than originally expected.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=4021982021-04-29T12:07:02Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Feedback</i></li></ul><p>PR: <a href="https://github.com/openSUSE/cpanspec/pull/29" class="external">https://github.com/openSUSE/cpanspec/pull/29</a></p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=4037682021-05-04T10:50:26Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Due date</strong> changed from <i>2021-04-30</i> to <i>2021-05-07</i></li></ul><p>PR is still running on the autoupdate host for further tests. Looking fine so far.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=4038252021-05-04T13:42:33Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Status</strong> changed from <i>Feedback</i> to <i>Resolved</i></li></ul><p>Merged PR <a href="https://github.com/openSUSE/cpanspec/pull/29" class="external">https://github.com/openSUSE/cpanspec/pull/29</a></p>
<p>Rolled out master on autoupdate host.</p>
QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=4038342021-05-04T13:51:05Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-3 priority-4 priority-default closed behind-schedule" href="/issues/92122">action #92122</a>: Improve dependency detection via MYMETA.json in cpanspec</i> added</li></ul> QA - action #90047: Improve dependency detection in cpanspechttps://progress.opensuse.org/issues/90047?journal_id=4038462021-05-04T13:52:47Ztinitatina.mueller+trick-redmine@suse.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-4 status-1 priority-4 priority-default" href="/issues/92119">action #92119</a>: Improve license detection in cpanspec</i> added</li></ul>