tickets #70018
Updated by hellcp over 4 years ago
Because I'm lazy, I pretty much scripted the entire process of importing the mailing lists on baloo with an account allowed to read subscriber lists ```sh #!/usr/bin/sh mkdir -p /tmp/transition ls /var/spool/mlmmj/ > /tmp/transition/mailing-lists for i in $(cat /tmp/transition/mailing-lists); do if [ -d "/srv/www/htdocs/$i" ]; then for j in /srv/www/htdocs/$i/*.mbox.gz; do gunzip -c $j >> /tmp/transition/$i.mbox done else for j in /var/spool/mlmmj/$i/archive/*; do cat /srv/www/htdocs/$i/$i-2020-11.mbox > /var/spool/mlmmj/$i/archive/$j >> /tmp/transition/$i.mbox done fi mlmmj-list -L /var/spool/mlmmj/$i /var/spool/$i > /tmp/transition/$i.subscribers mlmmj-list -L /var/spool/mlmmj/$i -n /var/spool/$i > /tmp/transition/$i.nomail mlmmj-list -L /var/spool/mlmmj/$i -o /var/spool/$i > /tmp/transition/$i.owners mlmmj-list -L /var/spool/mlmmj/$i -m /var/spool/$i > /tmp/transition/$i.moderators done ``` on a local machine for verification and to write $list.mapping file listing mailman list ids and $list.address file listing mailman list address, since mailing lists have to be set up manually, since mailman lacks a few essential settings when creating mailing lists by itself (we basically need to have them setup in postorius to be private, so relying on cli isn't the best of ideas) ```sh #!/usr/bin/sh rsync --info=progress2 -r hellcp@baloo.infra.opensuse.org:/tmp/transition ./ rsync --info=progress2 -r transition hellcp@mailman3.infra.opensuse.org:/tmp/ ``` on mailman3 with mailman user ```sh #!/usr/bin/sh for i in $(cat /tmp/transition/mailing-lists); do cd /var/lib/mailman/ mailman addmembers -W addmember -d regular /tmp/transition/$i.subscribers $(cat /tmp/transition/$i.mapping) mailman addmembers -W addmember -d disabled /tmp/transition/$i.nomail $(cat /tmp/transition/$i.mapping) cd /var/lib/mailman_webui/ python3 manage.py hyperkitty_import --no-sync-mailman --since 1990-01-01 -l $(cat /tmp/transition/$i.address) /tmp/transition/$i.mbox python3 /tmp/transition/modop.py $i done ``` and additional python script to add owners and moderators under /tmp/transition/modop.py referenced above ```python #!/usr/bin/python3 import sys import os import mailmanclient list_name = sys.argv[0] mapping = '' with open('/tmp/transition/' + list_name + '.mapping') as mapping_file: mapping = mapping_file.read().replace('\n', '') client = mailmanclient.Client("localhost:8001", "mailman", os.environ['MAILMAN_PASSWORD']) list = client.get_list(mapping) with open('/tmp/transition/' + list_name + '.owners') as owners: for owner in owners: list.add_owner(owner) with open('/tmp/transition/' + list_name + '.moderators') as moderators: for moderator in moderators: list.add_moderator(moderator) ``` The prototype mapping of the old to the new addresses (some of the addresses will only be imported for archives, and their backend mailing lists will be removed soon after) ``` admin@opensuse.org admin@lists.opensuse.org admin-auto@opensuse.org admin-auto@lists.opensuse.org board@opensuse.org board@lists.opensuse.org cfp@opensuse.org cfp@lists.opensuse.org donations@opensuse.org donations@lists.opensuse.org election-officials@opensuse.org election-officials@lists.opensuse.org forums-admin@opensuse.org forums-admin@lists.opensuse.org freighttrain@opensuse.org freighttrain@lists.opensuse.org gsoc-mentors@opensuse.org gsoc-mentors@lists.opensuse.org heroes@opensuse.org heroes@lists.opensuse.org ircops@opensuse.org ircops@lists.opensuse.org journalists@opensuse.org journalists@lists.opensuse.org kernel-bugs@opensuse.org kernel-bugs@lists.opensuse.org kubic-bugs@opensuse.org kubic-bugs@lists.opensuse.org libeconf-devel@opensuse.org libeconf-devel@lists.opensuse.org libyui-commit@opensuse.org libyui-commit@lists.opensuse.org maintenance@opensuse.org maintenance@lists.opensuse.org membership-officials@opensuse.org membership-officials@lists.opensuse.org mirror@opensuse.org mirror@lists.opensuse.org mysql-packaging@opensuse.org mysql-packaging@lists.opensuse.org news@opensuse.org news@lists.opensuse.org obs-commits@opensuse.org obs-commits@lists.opensuse.org obs-devel@opensuse.org obs-devel@lists.opensuse.org obs-errors@opensuse.org obs-errors@lists.opensuse.org obs-tests@opensuse.org obs-tests@lists.opensuse.org openqa-admin@opensuse.org openqa-admin@lists.opensuse.org opensuse@opensuse.org users@lists.opensuse.org opensuse-announce@opensuse.org announce@lists.opensuse.org opensuse-arm@opensuse.org arm@lists.opensuse.org opensuse-artwork@opensuse.org artwork@lists.opensuse.org opensuse-autoinstall@opensuse.org autoinstall@lists.opensuse.org opensuse-base@opensuse.org base@lists.opensuse.org opensuse-bugs@opensuse.org bugs@lists.opensuse.org opensuse-bugshare@opensuse.org bugshare@lists.opensuse.org opensuse-buildservice@opensuse.org buildservice@lists.opensuse.org opensuse-cloud@opensuse.org cloud@lists.opensuse.org opensuse-commit@opensuse.org commit@lists.opensuse.org opensuse-conference@opensuse.org conference@lists.opensuse.org opensuse-cz@opensuse.org users-cz@lists.opensuse.org opensuse-de@opensuse.org users-de@lists.opensuse.org opensuse-doc@opensuse.org doc@lists.opensuse.org opensuse-edu@opensuse.org edu@lists.opensuse.org opensuse-el@opensuse.org users-el@lists.opensuse.org opensuse-es@opensuse.org users-es@lists.opensuse.org opensuse-factory@opensuse.org factory@lists.opensuse.org opensuse-factory-mozilla@opensuse.org factory-mozilla@lists.opensuse.org opensuse-features@opensuse.org features@lists.opensuse.org opensuse-fr@opensuse.org users-fr@lists.opensuse.org opensuse-gnome@opensuse.org gnome@lists.opensuse.org opensuse-go@opensuse.org go@lists.opensuse.org opensuse-ha@opensuse.org ha@lists.opensuse.org opensuse-haskell@opensuse.org haskell@lists.opensuse.org opensuse-hu@opensuse.org users-hu@lists.opensuse.org opensuse-isdn-de@opensuse.org isdn-de@lists.opensuse.org opensuse-ja@opensuse.org users-ja@lists.opensuse.org opensuse-kde3@opensuse.org kde3@lists.opensuse.org opensuse-kde@opensuse.org kde@lists.opensuse.org opensuse-kde-bugs@opensuse.org kde-bugs@lists.opensuse.org opensuse-kernel@opensuse.org kernel@lists.opensuse.org opensuse-kubic@opensuse.org kubic@lists.opensuse.org opensuse-m17n@opensuse.org m17n@lists.opensuse.org opensuse-marketing@opensuse.org marketing@lists.opensuse.org opensuse-offtopic@opensuse.org offtopic@lists.opensuse.org opensuse-openstack@opensuse.org openstack@lists.opensuse.org opensuse-optimize@opensuse.org optimize@lists.opensuse.org opensuse-packaging@opensuse.org packaging@lists.opensuse.org opensuse-ppc@opensuse.org ppc@lists.opensuse.org opensuse-programming@opensuse.org programming@lists.opensuse.org opensuse-project@opensuse.org project@lists.opensuse.org opensuse-proofreading@opensuse.org proofreading@lists.opensuse.org opensuse-pt@opensuse.org users-pt@lists.opensuse.org opensuse-python@opensuse.org python@lists.opensuse.org opensuse-releaseteam@opensuse.org releaseteam@lists.opensuse.org opensuse-ruby@opensuse.org ruby@lists.opensuse.org opensuse-science@opensuse.org science@lists.opensuse.org opensuse-security@opensuse.org security@lists.opensuse.org opensuse-security-announce@opensuse.org security-announce@lists.opensuse.org opensuse-sl@opensuse.org users-sl@lists.opensuse.org opensuse-summit@opensuse.org summit@lists.opensuse.org opensuse-support@opensuse.org support@lists.opensuse.org opensuse-test@opensuse.org test@lists.opensuse.org opensuse-translation@opensuse.org translation@lists.opensuse.org opensuse-translation-commit@opensuse.org translation-commit@lists.opensuse.org opensuse-translation-de@opensuse.org translation-de@lists.opensuse.org opensuse-translation-el@opensuse.org translation-el@lists.opensuse.org opensuse-translation-es@opensuse.org translation-es@lists.opensuse.org opensuse-translation-fr@opensuse.org translation-fr@lists.opensuse.org opensuse-updates@opensuse.org updates@lists.opensuse.org opensuse-virtual@opensuse.org virtual@lists.opensuse.org opensuse-web@opensuse.org web@lists.opensuse.org opensuse-web-de@opensuse.org web-de@lists.opensuse.org opensuse-wiki@opensuse.org wiki@lists.opensuse.org opensuse-xfce@opensuse.org xfce@lists.opensuse.org opensuse-zsystems@opensuse.org zsystems@lists.opensuse.org press@opensuse.org press@lists.opensuse.org release@opensuse.org release@lists.opensuse.org review@opensuse.org review@lists.opensuse.org shop@opensuse.org shop@lists.opensuse.org sourcedvd@opensuse.org sourcedvd@lists.opensuse.org studio-express@opensuse.org studio-express@lists.opensuse.org travel-support@opensuse.org travel-support@lists.opensuse.org tweet-master@opensuse.org tweet-master@lists.opensuse.org ulp-devel@opensuse.org ulp-devel@lists.opensuse.org uyuni-announce@opensuse.org announce@lists.uyuni-project.org uyuni-devel@opensuse.org devel@lists.uyuni-project.org uyuni-users@opensuse.org users@lists.uyuni-project.org uyuni-translation@opensuse.org translation@lists.uyuni-project.org vagrant-admins@opensuse.org vagrant-admins@lists.opensuse.org video@opensuse.org video@lists.opensuse.org weblate-admin@opensuse.org weblate-admin@lists.opensuse.org wicked@opensuse.org wicked@lists.opensuse.org wicked-commit@opensuse.org wicked-commit@lists.opensuse.org wicked-devel@opensuse.org wicked-devel@lists.opensuse.org yast-bugs@opensuse.org yast-bugs@lists.opensuse.org yast-commit@opensuse.org yast-commit@lists.opensuse.org yast-community@opensuse.org yast-community@lists.opensuse.org yast-devel@opensuse.org yast-devel@lists.opensuse.org zypp-commit@opensuse.org zypp-commit@lists.opensuse.org zypp-devel@opensuse.org zypp-devel@lists.opensuse.org ``` This far only the uyuni guys decided to move over to their domain (as visible above), adrianS from OBS team stated they aren't currently interested For the external `$domain`s: * DNS entry: A and AAAA with the IP of proxy.o.o * MX DNS entries (mx*.o.o) * haproxy config * postfix config to ensure `$domain` mails get routed to mailman3