1) could it be that the script that uploads the results (where is this one running?) misses some sanity checks ?

Absolutely. I am digging into it during this morning. I am running all the process from scratch to detect the problem. I will update the issue with what I found.

2) why is this analyser not running directly on stage.opensuse.org ?

The current architecture is a bit more convoluted. There are four machines involved:

1.- Calloway. The old Cab Calloway is the only machine with rights to access the mirrordb database (the IP is in the postgresql white list). This machine runs, using crontab, the script 'mirror_brain', used to create the list of files (and sizes) candidates to be mirrored. Using 'crontab -e' we can see the entry:

0 3 * * * ( cd /suse/aplanas/Documents/09-rsyncd ; ./mirror_brain )

2.- Lena. My desktop. Is the one that wait Calloway and run the KP algorithm. Also is the one that create the 'payload' file from the logs files in langley. This is used to deduce the importance of every file. With the mirror_brain file and the payload file, KP can deduce the mirror files

3.- pontifex2-opensuse. There is a script running that takes care of sync the different directories that contains result of KP. The current setup was improved by coolo in December, but basically there is something like this:

ssh mirror@pontifex2-opensuse
ps aux | grep update-rsync-notify.py

The update-rsync-notify.py is in /srv/rsync-modules/ and need to be run like this

python /srv/rsync-modules/update-rsync-notify.py --src /srv/ftp-stage/pub/opensuse/ --srcalt /srv/ftp/pub/opensuse/ --dst /srv/rsync-modules/ --lists /srv/rsync-modules/result-lists/ 30g 80g 160g 320g 640g > /srv/rsync-modules/update-rsync.log

The service can be found in

/etc/sv/update-rsync-modules

4.- External rsync. I don't know this machine because was completely managed by coolo, but the setup is the same that pontifex2-opensuse.

3) langley has current logfiles, so why does it still not work ?

I will update this task with my findings. I am running the scripts in 1), 2) and 3) at hand now (the one in 2 is slow)

I'm out of the office until Thursday, 14th of April 2014.

During my absence, please contact

• autobuild@suse.de for all questions around Autobuild and the Build Service
• ops-services@suse.de for all questions around OPS related tasks
• Andreas Mach as my deputy for all other questions

With kind regards
Lars Vogdt

--
Lars Vogdt

• OPS Engineering Services Team Lead - SUSE Linux Products GmbH - GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer Maxfeldstraße 5, 90409 Nuernberg, Germany - HRB 16746 (AG Nuernberg)

04/16/14 09:27 >>>

[openSUSE Tracker]
Issue #2302 has been updated by aplanas.

After hitting me so bad, please, do not put this task into oblivion. Widehat still do not have the put-knapsack module.

Is this still relevant?

tickets #2302: Hotstuff modules empty on stage.opensuse.org
https://progress.opensuse.org/issues/2302#change-9080

• Author: lrupp
• Status: Feedback
• Priority: Urgent
• Assignee: aplanas
• Category: Mirrors
• Target version:

* Due subtask: 15/04/2014¶

All hotstuff modules on stage.opensuse.org are currently empty.

The update-rsync-notify.py script hangs in a wait state and currently does : nothing.

But all mirrors using our recommended rsync commandline are now also empty. That does not help to distribute the load. As result, I disabled the hotstuff modules now on rsync.opensuse.org and on stage.opensuse.org.

Coolo told me now, that the script is analysing the logs that are produced on stage.opensuse.org once they are synced to langley.

Now I have 3 questions/remarks:
1) could it be that the script that uploads the results (where is this one running?) misses some sanity checks ?
2) why is this analyser not running directly on stage.opensuse.org ?
3) langley has current logfiles, so why does it still not work ?

--
You have received this notification because you have either subscribed to it, or are involved in it.
To change your notification preferences, please click here: http://progress.opensuse.org/my/account