Differences between revisions 168 and 169
Revision 168 as of 2022-07-04 23:23:12
Size: 18438
Editor: NickBannon
Comment: Update netboot.xyz
Revision 169 as of 2022-07-12 14:59:58
Size: 18481
Editor: GaryODonovan
Comment: Power sockets on mobo of catfish have the reliability of cat piss, if it gives trouble again reseat these connections.
Deletions are marked like this. Additions are marked like this.
Line 8: Line 8:
||[GPO][BRD]||2022-07-11||Fixed catfish||

UCC Systems Change Log

Please enter a short description of any work performed on UCC systems here.

Whom

When

What

[GPO][BRD]

2022-07-11

Fixed catfish

[MPT][MTL][NTU]

2022-07-04

Rebuild cloud-mooneye

[MTL][NTU]

2022-07-04

Update netboot.xyz on murasoi

[MTL]

2022-07-02

Upgrade to latest gitlab

[MPT][GPO][BRD]

2022-06-27

Fixed square payments on uccportal

[MTL]

2022-06-21

Fix mail on mollitz

[NTU][TPG][GPO]

2022-06-20

Cleared space on mollitz to allow for backups again

[MPT]

2022-06-13

Shut down most services on mooneye and migrate to more appropriate locations

[MTL][NTU]

2022-06-06

Rebuild mudkip

[NTU][MTL]

2022-03-14--2022-03-15

ucc-ansible-soe fully rebuild idle-mudkip, idle-magikarp, now Debian 11

[NTU][MTL]

2022-03-07

ucc-ansible-soe fully rebuild idle-maltair, idle-medico, now Debian 11

[MPT]

2022-03-03

Git-ify and modernise more of the club website

[MTL]

2022-02-28

Upgrade to latest gitlab

[MTL]

2022-02-28

Additional packages in UCC Ansible SOE for idlehosts

Chris+[DBA][MPT][NTU]

2022-01-24

reracking machop, DELL R710 x 2, ...

[NTU]

2022-01-24

motsugo:/tmp full

[MTL][NTU]

2022-01-10

quovadis.ucc.asn.au (self-serve letsencrypt) moved to using cloud-mooneye and RFC-2136

[MTL][NTU][TPG][333]

2022-01-10

mudkip dead fan 7, degraded fan 1: replaced, now boots happily

[MTL][NTU]

2021-12-09

uccmonitor cloud-alchemy grafana upgrade

[333]

2021-11-14

Implemented new Tech Todo List in GitLab to manage our tech/wheel action items better(er)*

[MTL][NTU][BOB]

2021-11-01

machop setup; start proxmox atlantic cluster upgrade v6.3 to v6.4; ansible-driven idle-machop SOE

[BRD][MTL][NTU][BOB][TPG] [TBB][333][pcadmin] [DTA][MDD][SJY][RME]

2021-10-05

power outage 03:18 - recovery!, machop setup

[MTL][NTU][BOB][TPG][333]

2021-10-04

proxmox docs, discord-irc restarts, uccmonitor, magikarp recovery

[MPT][NTU]

2021-09-27

uccportal renewals analysis; update and make agenda.default sourceable/executable

[TPG]

2021-09-27

SSL/TLS cert for eggman

[MTL][NTU]

2021-09-27

Test prometheus/grafana monitoring of SMART on mooneye, motsugo

[BOB]

2021-08-31

Fixed apt sources on samson (was waia) and updated

[BOB]

2021-07-28

Free up space on maaxen and (manually) upgrade to latest available windows updates. Disabled print spooler to prevent printnightmare vulnerabilities

[MPT]

2021-06-30

Create wheel-runner VM and configure Gitlab CD for ucc-fw

[MPT][NTU]

2021-05-17

ucc-announce subscription backlog from ucc-adduser-ad

[TRS]

2021-05-10

Fix BigBlueButton, including upgrading to 2.3.0 release

[MTL]

2021-05-07

Fix webdispense

[MTL]

2021-05-06

Fix full /var on uccportal, add logrotate

[BOB]

2021-04-30

Fixed radiusd config on samson to suit updated requirements for full hostname and IP

[BOB][TPG][NTU]

2021-04-22

Fixed our long-broken IPSec VPN: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/2021-April/005460.html

[333]

2021-04-20

Fixed MariaDB on Mussel, and thus OCSInventory - which depends on it

[333]

2021-04-18

Completely rebuilt the idle-medico VM using only the Ansible playbook that [MTL] wrote

[TPG]

2021-03-08

Fixed gitlab certbot

[MTL]

2021-02-22

Implement age command in uccpass. Migrate uccpass repo to UCC gitlab

[MTL]

2021-02-20

Move octoDNS to dedicated venv for UCC Cloudflare DNS sync

[MTL]

2021-01-20

Poke certs for irc.ucc.asn.au

[MPT]

2021-01-18

Remove mollitz mock internal subnet from murasoi

[MPT][NTU]

2021-01-18

Tinker with mollitz config and update wiki page to match

[TRS][MPT]

2021-01-12

Return mollitz legacy backups, offsite

[TPG]

2021-01-11

molmol freebsd-update

[NTU]

2021-01-10

atlantic cluster magikarp extend ceph osd.3

[BOB]

2021-01-10

atlantic cluster apt updates, reboots

[TPG][NTU]

2020-12-28,30

mailfish:~spamass training

[333][NTU]

2020-12-21

Replace molmol mirror SSD: root:geom-mirror boot swap zfs-slog

[TPG]

2020-12-21

webcam archiving and retrieval optimisation

[NTU]

2020-12-11

Test Ceph WAL on vmstore-bigssd, add discord-irc to legacy backups

[NTU][MTL]

2020-12-10

mailfish: Bounce due to NFS problems

[MPT]

2020-12-02

Fix Abe configuration and rancid

[MPT]

2020-12-02

Troubleshoot UniSFA TV network connection

[MPT]

2020-11-30

Reshuffle UCC wifi channels

[NTU][MTL]

2020-11-29

mussel: fix MoinMoin wiki textcha again

[NTU][CJS][MPT]

2020-10-19

place mollitz (legacy backups) on UCC network

[MPT]

2020-10-19

Create user VM for [DIE] for SS13 server

[MTL][MPT][DAA]

2020-10-12

Make login.ucc.asn.au work

[MPT][333][NTU]

2020-10-09--2020-10-12

CEPH! (Fix latency issues, lots of tweaks to Ceph config...)

[MTL]

2020-10-06

Fix gitlab.ucc.asn.au LE certs

[MTL]

2020-10-06

Fix portal.ucc.asn.au LE certs on uccportal. Clear full /tmp and /var partitions

[MTL]

2020-08-31

Fix webmail.ucc.asn.au redirect. Curse mussel apache config

[TPG]

2020-08-29

Fixed the wheel group page

[MTL]

2020-08-21

Fix wildcard certs on mailfish, mussel and motsugo via ucc-ansible-soe and Cloudflare API key

[MTL]

2020-07-28

Fix mentions of mooneye in mailfish config vis headers and spam

[MTL]

2020-06-08

Fix HTTP redirect to HTTPS for the wiki

[MTL]

2020-06-02

Add LDAP configuration for UCC AD to uccmonitor

[MPT]

2020-05-27

Enable nested KVM virtualisation on mudkip, check enabled on others

[NTU]

2020-05-24

lvextend -r murasoi:/var, add /etc/logrotate.d/ucc-ssh-auth

[BOB]

2020-04-25

dist-upgrade motsugo to buster

[BOB]

2020-04-25

Renewed letsencrypt certs on murasoi to fix ipsec vpn

[BOB]

2020-04-25

Changed main NS record with DNS registrar from mooneye.ucc.gu.uwa.edu.au to mooneye.ucc.asn.au, updated zone records to suit, and changed _acme-challenge.secure.ucc.asn.au zone to use mooneye.ucc.asn.au

[BOB]

2020-04-24

Fix rancid issues introduced by upgrade

[BOB]

2020-04-21

dist-upgrade mussel to buster

[BOB]

2020-04-21

dist-upgrade gitlab to buster

[MTL]

2020-04-21

dist-upgrade meersau to buster; rejoin AD and fix Xrdp logins - see https://github.com/neutrinolabs/xrdp/issues/906

[MPT]

2020-04-18

L3 setup of 4G modem, enabled policy-based routing for murasoi

[MPT]

2020-04-17

Physical and L2 setup of 4G modem in machine room

[MPT]

2020-04-17

Install replacement fans and recommission mudkip

[NTU][MTL]

2020-04-15

Surgical update of mussel's DebianPkg:libapache2-mod-wsgi to 4.5.17-1_i386.deb to fix moinmoin 866180

[MTL]

2020-04-13

Deploy FoldingAtHome to UCC clubroom machines to keep them busy during the COVID19 pandemic

[MTL]

2020-04-11

Migrate UCC wiki from mooneye to mussel

[NTU][TRS]

2020-04-09

Add mollitz (offsite backups) to monitoring

[MTL]

2020-04-06

Set up Mailauesi with haproxy for IMAPS, POP3S and SMTP submission frontend load balancer

[TRS]

2020-03-21

Fixed the VPN by pointing it at the current let's encrypt cert

[TRS]

2020-03-21

Installed BigBlueButton

[MSH]

2020-03-21

Make motsugo logrotate btmp (failed auth attempts) weekly so disk doesn't fill up

[DAS][MPT]

2020-02-26

Troubleshooting Unigames webcams, replugged loose patch cable, moved VLAN1 to new Lard trunk

[MPT][CFE]

2020-02-25

Increase 5G channel width to 40MHz into free space

[MPT][NTU]

2020-02-24

Shutdown Magikarp and diagnose fan issue

[MSH]

2020-02-15

Set mooneye:/etc/opendkim.conf "MaximumHeaders 0" to handle >64KB headers from looping mail to/from(?) [email protected]

[MPT]

2020-02-10

Convert all PVEVMUser permissions to newer UCC_VM_User, update documentation

[BOB]

2020-02-10

Audited user VM's

[BOB]

2020-01-28

Change ocsinventory to ssl only and roll out to all machines

[MPT]

2020-01-27

Resize /dev/md0 and /var on murasoi, with room to grow

[MPT]

2020-01-27

Fix uccpass regression preventing new keys being added

[TEC][NTU][MTL]

2020-01-13

Install Prometheus/Grafana monitoring, with Ansible, on uccmonitor, resize /var, add custom metric

[TEC]

2019-12-24

Install Prometheus/Grafana monitoring, with Ansible, on uccmonitor

[MPT][NTU]

2019-12-{16,17}

Provision new RPi 4 as user server titan, install SOE and useful programs

[MPT]

2019-12-12

Migrate UCC mail to new IronPort cloud appliances with UWA

[MPT][NTU]

2019-12-09

Outbound mail relayhost hostname {antivirus -> smtp}.uwa.edu.au per UniIT request

[NTU][TBB]

2019-12-07

merlo /usr/local/bin/serial-connect fixme resets VTxxx terminal

[MPT]

2019-12-02

Improved network documentation somewhat

[MPT][NTU][MTL]

2019-12-02

Provisioned new "IoT devices" network, including L2, L3 and PSK wifi

[NTU][MPT]

2019-12-02

Created an AWS account for UCC

[MPT][NTU]

2019-11-25

Fix cobra networking (broken small switch), /away automounting

[TEC][333]

2019-11-25

add magikarp and mudkip to Atlantic proxmox cluster

[TEC][MDD]

2019-11-25

update minecraft2017 -> minecraft2019

[333]

2019-11-21

Diagnose and fix smallwing's failing UniFi firmware updates

[MPT][TPG]

2019-11-16

Unborking a user account, restrict usernames to 3 or more characters

[MPT]

2019-11-05

Redirected fresher@ to rest of committee-only@ (successfully this time)

[MSH]

2019-10-22

Set kernel.dmesg_restrict=0 on motsugo and mussel

[MPT][THA][TPG][MSH][NTU]

2019-10-18

Service troubleshooting and restarts, after 0430--0521 power outage: proxmox/samba/ssd/zonemake?

[FVP]

2019-09-20--2019-10-11

username change?

[MTL]

2019-08-22

dpkg --add-architecture i386; apt update; apt install libpcre2-dev:i386 libncurses5-dev:i386 on motsugo for [DAA]. Also, free up space on full /var

[MPT]

2019-08-14

Implemented soft-blacklist for possibly-broken account names in ucc-adduser-ad

[FVP]

2019-08-13

Replaced faulty Microtik router sharpchin with spare OpenWRT device coromandel (in UniSFA)

[MPT]

2019-08-09

Changed ucc-adduser-ad to make new users' index.html readable by the webserver

[MPT]

2019-07-31

Updated MOTD to reflect [DBA]'s resignation from committee

[MPT]

2019-07-29

Removed ex-group members from their old groups on UCC systems (following group review)

[DAA]

2019-07-29

Tried and failed to upgrade Guacamole to 1.0.0 (using Postgres + RADIUS and then Postgres + LDAP), rolled back to 0.9.14

[333][NTU][TPG]

2019-07-22

Wifi - fix smallwing/nanoHD VLAN connectivity to kerosene, was curviceps

[MPT]

2019-07-20

Enabled daily updates of Linux desktops

[MPT]

2019-07-20

Created 90-ucc-desktops.conf for PolKit and deployed to all desktops, enabled unattended upgrades on all desktops

[MPT]

2019-07-18

Joined Catfish's Linux install to the domain, fixed network logins

[TPG][NTU]

2019-07-15

Switch complications?

[TRS][NTU]

2019-07-14

UWA DNS outage, check bind+dd/discord-irc, disable/enable mooneye DNS forwarders

[FVP][MPT][DAS]

2019-07-12

Replaced machine room switch with new chassis and supervisor with 10GE capability

[DAA]

2019-07-11

Enlarged /backups array, partition, logical volume and filesystem on Mollitz

[DAA]

2019-07-09

Installed FreeBSD updates to 11.2-RELEASE-p11 on Molmol & upgraded pkg set

[MPT]

2019-06-28

Reinstalled clownfish with updated Mint 19.1 on the new NVMe SSD

[BOB]

2019-06-25

Changed door mailing list settings to discard spam emails

[BOB][MPT][FVP][AJT][TRS]

2019-06-21--2019-06-22

Recovered murasoi from dual system disk failure

[MSH]

2019-06-18

Filtered SACK on murasoi ucc-fw to avoid SACK Panic

[MPT]

2019-06-10

Fixed /etc/fstab on pinball to boot new NVMe SSD

[MPT]

2019-06-10

Swapped in new fans to pinball and clownfish (with [AJT])

[MPT]

2019-06-09

Fixed network issues on upgraded clownfish

[AJT]

2019-06-07

Upgraded pinball, clownfish and porcupine with new components (with [MPT])

[DAA]

2019-05-27

Upgraded Samba on Samson from 4.9.2 to 4.9.5

[MTL]

2019-05-20

Some housekeeping done on mooneye /etc/aliases. All mailman mailing lists moved to bottom of file in prep for later mailman upgrade

[DAA]

2019-05-18

Removed a stack of obsolete Python packages that were breaking things

[DAA]

2019-05-18

Gitlab maintenance - fixed UCC user accounts to be internal, not external, and removed heaps of spam accounts

[DAA]

2019-05-15

Converted Apache on Mussel to Certbot and wildcard SSL; removed the acme-members scripts from crontab and zonemake; converted Dovecot on Motsugo to Certbot and wildcard SSL; converted IPsec on Murasoi to use DNS challenges; dropped acmetool on Mussel, Mooneye, Motsugo, Murasoi and Gitlab; wrote SSLCertificates page

[DAA]

2019-05-15

Security patches installed for Molmol, Meersau and Maaxen

[DAA]

2019-05-13

Converted inspircd on Mussel to Certbot

[DAA]

2019-05-13

Converted postfix and apache on Mooneye to Certbot and wildcard SSL

[DAA]

2019-05-12

Set up infrastructure for wildcard SSL certificates

[BOB][DBA]

2019-05-03

Installed replacement system/slog disk SSD in molmol

[DAA]

2019-05-03

Removed a large number of obsolete packages (mostly libraries and old development tools) from Mussel

[DAA]

2019-05-02

Upgraded Molmol to FreeBSD 11.2-RELEASE
Upgraded Samba to 4.8 (supported version) on molmol

[FVP]

2019-05-01

Refactored uccpass to put user keys in separate files, with committee and wheel directories

[MPT]

2019-04-30

Removed elliotnunn from local sudo /etc/group on clownfish to stop Polkit from asking everyone for his password

[MPT]

2019-04-30

Added [MLG] to committee user group, and updated motd

[DAA]

2019-04-29

Upgraded to Debian 9.9 on Mussel, Motsugo, Murasoi, Mooneye, Merlo, Samson, Mollitz, Gitlab, idle-Maltair, idle-Medico, Heathred, Medico, Maltair and Loveday; ceph restarted and checked for status

[BOB]

2019-04-27

Updated salmon and installed unattended upgrades. Sorted through VM cluster and tagged a heap of VM's for deletion

[BOB]

2019-04-26

Specced and purchased replacement+upgrade disks for molmol

[MSH]

2019-04-23

Made DNS notifies/AXFR from secondaries work

[MSH]

2019-04-23

Fixed member web subdomains and ssl

[DAA]

2019-04-20

Removed a bunch of obsolete packages from motsugo (gcc-4.6 ruby1.8 ruby1.9.1 php5-cli postgresql-client-9.4 python2.6-minimal zlib-bin libjasper1 hardening-includes python-reportbug python-support sysvinit)
upgraded the backport of lepton
upgraded Docker to a supported version

[BEN]

2019-04-18

Setup unisfa-freshers mailing list

[NTU][TEC]

2019-04-15

ocsinventory-agent deployed to corvo, napoli

[BOB]

2019-04-13

Set up ocsinventory.ucc.asn.au and deployed to most machines

[DAA]

2019-04-12

Upgraded idle-{medico,maltair} to Debian 9; installed Docker and Gitlab Runner on idle-medico

[DAA]

2019-04-10

Installed rust and cargo from testing on Motsugo

[SJH][MSH]

2019-04-09

Fixed user php7

[DAA]

2019-04-09

Updated Roundcube to the version in Debian

[BOB]

2019-04-09

Disabled php5 and enabled php7 on mussel

[DAA]

2019-04-09

Attempted to fix eggman mounting /mp3s on boot by enabling systemd-networkd-wait-online.service

[DAA]

2019-04-09

Removed motsugo pip packages for Pygments, Six, and youtube-dl as they were old and newer Debian packages were installed

[MTL]

2019-04-03

mussel package upgrades

[MSH]

2019-04-02

Removed and readded motsugo sdc and sde. Got rid of current-pending-sectors by overwriting them

[NTU] [MPT]

2019-04-01

Merge merlo:/var/log/dispense into ~coke/cokelog , restart rsyslogd on merlo, motsugo

[MPT]

2019-03-27

Remounted /home and /away on merlo to fix BAD PIN message

[BOB]

2019-03-26

Re-synced time on Loveday - clock drift was causing ceph to be unhealthy

[BOB]

2019-03-26

Installed LSI SAS card in new HP DL380 (maltair), updated assorted firmwares, fixed network interfaces and re-added to VM cluster

[DTA]

2019-03-26

Set passwords for flame users id, dos, linux - it might stop the problem with aserver.err filling up after a portscan

[MSH]

2019-03-23

put semrushbot in secure.ucc, wiki.ucc robots.txt (not sure if wiki worked

[NTU] [MPT]

2019-03-18

reset napoli passwd, add sudoers

[MSH]

2019-03-18

Got arpwatch running again on murasoi, had to put br0.* in /etc/arpwatch.conf

[NTU] [TPG]

2019-03-16

merlo not remote logging and BAD PIN? reboot, remount home, restart rsyslogd on merlo, motsugo

[NTU] [FVP] [JGM]

2019-03-16

Server rack in corridor, temporary? switch to VT320 with a narrower keyboard until a narrow PS/2 keyboard is found for the VT510 https://lists.ucc.gu.uwa.edu.au/pipermail/tech/2019-March/005091.html

[MSH]

2019-03-16

Created mooneye:/var/log/journal so logs will persist across reboots

[RME]

2019-03-15

Created this page. Let's see if it sticks.

[BOB]

2019-03-14

Removed root keys of wheel members who are not current members

[BOB]

2019-03-05

Attended Ross's auctions to inspect HP DL380 Gen 8 server for maltair replacement. Bid and won in auction. Delivered to UCC and installed

[TRS]

2019-02-05

Pointed SOGo to AD (was still LDAP.)

[TRS]

2018-12-01

Upgraded SOGo on mussel to 4.0.4.

[FVP] [CFE]

2018-11 to 2019-02

Created portal.ucc.asn.au based on [DAA]'s MemberDB for membership information