https://progress.opensuse.org/https://progress.opensuse.org/themes/openSUSE/favicon/favicon.ico?15829177842019-09-09T22:51:13ZopenSUSE Project Management ToolopenSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2422762019-09-09T22:51:13Zjberryjberry@suse.com
<ul><li><strong>Private</strong> changed from <i>Yes</i> to <i>No</i></li></ul> openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2422792019-09-09T23:00:09Zjberryjberry@suse.com
<ul></ul><pre><code>73509:2019-09-05 07:52:18 <1> pontifex2(20265) [zypper:req++] SolverRequester.cc(remove):358 Marking for deletion: I__s_o(832)tumbleweed-snapshot-0.4.3-3.1.noarch(@System)
73521:2019-09-05 07:52:18 <1> pontifex2(20265) [zypp] SATResolver.cc(resolvePool):719 Delete I_Tu_o(832)tumbleweed-snapshot-0.4.3-3.1.noarch(@System)
74178:2019-09-05 07:52:18 <1> pontifex2(20265) [libsolv] PoolImpl.cc(logSat):121 job: user installed tumbleweed-snapshot
74243:2019-09-05 07:52:18 <1> pontifex2(20265) [libsolv] PoolImpl.cc(logSat):121 job: erase tumbleweed-snapshot-0.4.3-3.1.noarch
74245:2019-09-05 07:52:18 <1> pontifex2(20265) [libsolv] PoolImpl.cc(logSat):121 !tumbleweed-snapshot-0.4.3-3.1.noarch [832]I (w1)
74263:2019-09-05 07:52:18 <1> pontifex2(20265) [zypper++] Summary.cc(readPool):127 <uninstall> I_Tu_ou(832)tumbleweed-snapshot-0.4.3-3.1.noarch(@System)
74303:2019-09-05 07:52:20 <1> pontifex2(20265) [zypp] RpmDb.cc(doRemovePackage):2139 RpmDb::doRemovePackage(tumbleweed-snapshot-0.4.3-3.1.noarch,0x00000008)
74304:2019-09-05 07:52:20 <1> pontifex2(20265) [zypp++] ExternalProgram.cc(start_program):249 Executing 'rpm' '--root' '/' '--dbpath' '/var/lib/rpm' '-e' '--allmatches' '--nodeps' '--' 'tumbleweed-snapshot-0.4.3-3.1.noarch'
74307:2019-09-05 07:52:20 <1> pontifex2(20265) [Progress++] ProgressData.cc(report):88 {#7|Removing tumbleweed-snapshot-0.4.3-3.1.noarch}END
</code></pre>
<p>The <code>openSUSE:infrastructure</code> no longer able to download metadata. Heading out so as far as I investigated.</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2422822019-09-09T23:00:28Zjberryjberry@suse.com
<ul><li><strong>Subject</strong> changed from <i>tumbleweed-snapshot package nukes from download.o.o machine</i> to <i>tumbleweed-snapshot package nuked from download.o.o machine</i></li></ul> openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2424382019-09-10T12:18:51Zdimstardimstar@opensuse.org
<ul></ul><p>So, it seems it was 'explicitly' removed, not accidentally:</p>
<pre><code>2019-09-05 07:52:20|command|root@pontifex2|'zypper' 'rm' 'tumbleweed-snapshot'|
# 2019-09-05 07:52:20 tumbleweed-snapshot-0.4.3-3.1.noarch removed ok
# Warning: Stopping tumbleweed-snapshot.service, but it can still be activated by:
# tumbleweed-snapshot.timer
# warning: /etc/tumbleweed-snapshot.conf saved as /etc/tumbleweed-snapshot.conf.rpmsave
2019-09-05 07:52:20|remove |tumbleweed-snapshot|0.4.3-3.1|noarch|root@pontifex2|
</code></pre>
<p>and yeah - fun stuff: the journal starts pretty much right after that only:</p>
<pre><code># journalctl
-- Logs begin at Thu 2019-09-05 08:19:26 UTC, end at Tue 2019-09-10 12:15:01 UTC. --
</code></pre>
<p>making it hard to see WHO performed the action</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2424442019-09-10T12:25:27Zdimstardimstar@opensuse.org
<ul></ul><p>right after the rm, a zypper dup was called - and the first user to login after the system was back up (very likely post update verification!?!) came from</p>
<p>Sep 05 08:33:34 pontifex2 sshd[17781]: Accepted publickey for root from 192.168.253.203 port 48722 ssh2: RSA SHA256:nYkj0QLB0lNAkMEfYXFsDQRdB6P2HVitZef6OZRHyv4</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2424922019-09-10T13:32:29Zjberryjberry@suse.com
<ul></ul><p>Reinstalled</p>
<pre><code>zypper in tumbleweed-snapshot
</code></pre>
<p>and reinstated config</p>
<pre><code>SOURCE_DIR="/srv/ftp/pub/opensuse"
SNAPSHOT_DIR="/srv/ftp/pub/opensuse/history"
SNAPSHOT_COUNT_MAX=20
RSYNC_INCLUDE="/usr/share/tumbleweed-snapshot/rsync/include-tumbleweed-only-repo.txt"
SOURCE_LOCK="/home/mirror/var/run/withlock/LOCK-publish"
</code></pre>
<p>enabled timer</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2425582019-09-10T16:10:31Zcboltzsuse-beta@cboltz.de
<ul></ul><blockquote>
<p>and yeah - fun stuff: the journal starts pretty much right after that only: [...]<br>
making it hard to see WHO performed the action</p>
</blockquote>
<p>/var/log/messages-*.xz exists ;-)</p>
<blockquote>
<p>Reinstalled [...]</p>
</blockquote>
<p>Now that you have written down how to setup tumbleweed-snapshot, what about moving that note to salt? ;-) (should be easy, but if needed, I can help - just ping me on IRC)</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2425612019-09-10T16:28:54Zjberryjberry@suse.com
<ul></ul><p>It was written in the original p.o.o issue as well. And short of custom config rather self documented by providing a proper package unlike 99% of the rest of the contents of the box.</p>
<p>I tracked down the SSH key used just before to mcaj@nibbler from authorized_keys file on root</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2430292019-09-11T19:43:51Zjberryjberry@suse.com
<ul></ul><p>Who knows who that is and so we can have a discussion to understand what happened and avoid this from happening again?</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2430832019-09-12T07:06:03Zduncregjgm@myself.com
<ul></ul><p>jberry wrote:</p>
<blockquote>
<p>Who knows who that is and so we can have a discussion to understand what happened and avoid this from happening again?</p>
</blockquote>
<p>Martin Caj.</p>
<p><a href="https://cz.linkedin.com/in/martin-caj-36a03399" class="external">https://cz.linkedin.com/in/martin-caj-36a03399</a></p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2433622019-09-13T12:03:35Zmcajmcaj@suse.com
<ul><li><strong>Assignee</strong> set to <i>mcaj</i></li></ul> openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2433652019-09-13T12:15:58Zmcajmcaj@suse.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>30</i></li></ul><p>Hi Jimmy,</p>
<p>Yes I worked on the machine recently. The machine has old<br>
unsupported OS. I did the updat, but there ware some problematic dependencies..<br>
During the work on it I probably removed the tumbleweed-snapshot package. <br>
For That I'm sorry. <br>
I checked the machine one more time today and it looks good.</p>
<p>Can you please check the server functionality and let me know if there is any problem with<br>
it ?</p>
<p>Thank you.<br>
Martin</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2433892019-09-13T15:36:52Zjberryjberry@suse.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>Assignee</strong> deleted (<del><i>mcaj</i></del>)</li></ul><p>Everything appears to be functioning at the moment. If there are dependency issues with the package file an issue and I'll take care of them, but tumbleweed-snapshot has very minimal dependencies. Removing packages, especially ones with the name of an openSUSE product in them on a openSUSE download server seems a bit haphazard to be done without investigating to see systemd services of the same name as well.</p>
openSUSE admin - tickets #56690: tumbleweed-snapshot package nuked from download.o.o machinehttps://progress.opensuse.org/issues/56690?journal_id=2433952019-09-13T15:44:56Zjberryjberry@suse.com
<ul></ul><p>To be clear the dependencies of tumbleweed-snapshot:</p>
<p>Requires: coreutils<br>
Requires: findutils<br>
Requires: grep<br>
Requires: procmail<br>
Requires: rsync</p>
<p>That's absolutely nothing. Given an explicit <code>zypper rm tumbleweed-snapshot</code> was run I find this whole thing fishy.</p>