QA » openQA Project

I retired jobs on OSD and couldn't pin-down the issue. Even if the directory already exists it works (as mentioned in the previous comment). If the target already exists but is a file I get

  Unable to copy '/var/lib/openqa/testresults/07630/07630513-sle-15-SP4-Online-ppc64le-Build61.1-create_hdd_sles4sap_gnome@ppc64le-2g' to '/var/lib/openqa/archive/testresults/07630/07630513-sle-15-SP4-Online-ppc64le-Build61.1-create_hdd_sles4sap_gnome@ppc64le-2g': Not a directory at /usr/share/openqa/script/../lib/OpenQA/Schema/Result/Jobs.pm line 270.

which is a different error (and besides leaves the question why there would be a file).

I can provoke the error message by creating a file (e.g. foo9) and then specifying a directory within as destination:

  DB<18> $res = File::Copy::Recursive::dircopy("foo1", "foo9/foo10");

  DB<19> print "error: $!";
error: File exists

But that would mean e.g. /var/lib/openqa/archive/testresults/07674 was a file in our case. Or: The the copying code noticed that the directory was missing, then a parallel job created it, then the copying code tried to create the directory but ran into the error because it now already exists. If that's correct we could solve the problem by simply creating the common-prefix directory (e.g. 07674) ourselves before invoking dircopy.

Weird. Maybe we could add some debugging like:

if ($! == 17) {
    # EEXISTS
    $errmsg .= qx{ls -l $dir_we_are_interested_in}
}

Looks like the code in File::Copy::Recursive is indeed broken. The module implements its own function to make a path (pathmk) and it does it like this:

mkdir($pth,$DirPerms) or return if !-d $pth && !$nofatal;

So if the directory doesn't exists when checking it but exists when trying to create it we'd get exactly the error we see¹ and since we indeed run those jobs concurrently that seems very possible.

This PR should fix it: https://github.com/os-autoinst/openQA/pull/4416

¹

  DB<29> print mkdir foo-new;
1
  DB<30> print mkdir foo-new;
0
  DB<31> print "error: $!";
error: File exists

mkittler wrote:

Looks like the code in File::Copy::Recursive is indeed broken. The module implements its own function to make a path (pathmk) and it does it like this:

mkdir($pth,$DirPerms) or return if !-d $pth && !$nofatal;

So if the directory doesn't exists when checking it but exists when trying to create it we'd get exactly the error we see¹ and since we indeed run those jobs concurrently that seems very possible.

This PR should fix it: https://github.com/os-autoinst/openQA/pull/4416

Yes, it should. Mojo::File reuses the core Perl functionality which can handle parallel path creation. https://metacpan.org/dist/File-Path/source/lib/File/Path.pm#L232