16817
Comment: recovery - portal/uccportal
|
17817
ucc-ansible-soe fully rebuild idle-maltair, idle-medico
|
Deletions are marked like this. | Additions are marked like this. |
Line 8: | Line 8: |
||[NTU][MTL]||2022-03-07||ucc-ansible-soe fully rebuild idle-maltair, idle-medico, now Debian 11|| ||[MPT]||2022-03-03||Git-ify and modernise more of the club website|| ||[MTL]||2022-02-28||Upgrade to latest gitlab|| ||[MTL]||2022-02-28||Additional packages in UCC Ansible SOE for idlehosts|| ||Chris+[DBA][MPT][NTU]||2022-01-24||reracking machop, DELL R710 x 2, ...|| ||[NTU]||2022-01-24||motsugo:/tmp full|| ||[MTL][NTU]||2022-01-10||quovadis.ucc.asn.au (self-serve letsencrypt) moved to using cloud-mooneye and RFC-2136|| ||[MTL][NTU][TPG][333]||2022-01-10||mudkip dead fan 7, degraded fan 1: replaced, now boots happily|| ||[MTL][NTU]||2021-12-09||uccmonitor cloud-alchemy grafana upgrade|| ||[333]||2021-11-14||Implemented new Tech Todo List in [[https://gitlab.ucc.asn.au/UCC/tech-todo-list|GitLab]] to manage our tech/wheel action items better(er)*|| ||[MTL][NTU][BOB]||2021-11-01||machop setup; start proxmox atlantic cluster upgrade v6.3 to v6.4; ansible-driven idle-machop SOE|| |
UCC Systems Change Log
Please enter a short description of any work performed on UCC systems here.
Whom |
When |
What |
[NTU][MTL] |
2022-03-07 |
ucc-ansible-soe fully rebuild idle-maltair, idle-medico, now Debian 11 |
[MPT] |
2022-03-03 |
Git-ify and modernise more of the club website |
[MTL] |
2022-02-28 |
Upgrade to latest gitlab |
[MTL] |
2022-02-28 |
Additional packages in UCC Ansible SOE for idlehosts |
Chris+[DBA][MPT][NTU] |
2022-01-24 |
reracking machop, DELL R710 x 2, ... |
[NTU] |
2022-01-24 |
motsugo:/tmp full |
[MTL][NTU] |
2022-01-10 |
quovadis.ucc.asn.au (self-serve letsencrypt) moved to using cloud-mooneye and RFC-2136 |
[MTL][NTU][TPG][333] |
2022-01-10 |
mudkip dead fan 7, degraded fan 1: replaced, now boots happily |
[MTL][NTU] |
2021-12-09 |
uccmonitor cloud-alchemy grafana upgrade |
[333] |
2021-11-14 |
Implemented new Tech Todo List in GitLab to manage our tech/wheel action items better(er)* |
[MTL][NTU][BOB] |
2021-11-01 |
machop setup; start proxmox atlantic cluster upgrade v6.3 to v6.4; ansible-driven idle-machop SOE |
[BRD][MTL][NTU][BOB][TPG] [TBB][333][pcadmin] [DTA][MDD][SJY][RME] |
2021-10-05 |
power outage 03:18 - recovery!, machop setup |
[MTL][NTU][BOB][TPG][333] |
2021-10-04 |
proxmox docs, discord-irc restarts, uccmonitor, magikarp recovery |
[MPT][NTU] |
2021-09-27 |
uccportal renewals analysis; update and make agenda.default sourceable/executable |
[TPG] |
2021-09-27 |
SSL/TLS cert for eggman |
[MTL][NTU] |
2021-09-27 |
Test prometheus/grafana monitoring of SMART on mooneye, motsugo |
[BOB] |
2021-08-31 |
Fixed apt sources on samson (was waia) and updated |
[BOB] |
2021-07-28 |
Free up space on maaxen and (manually) upgrade to latest available windows updates. Disabled print spooler to prevent printnightmare vulnerabilities |
[MPT] |
2021-06-30 |
Create wheel-runner VM and configure Gitlab CD for ucc-fw |
[MPT][NTU] |
2021-05-17 |
ucc-announce subscription backlog from ucc-adduser-ad |
[TRS] |
2021-05-10 |
Fix BigBlueButton, including upgrading to 2.3.0 release |
[MTL] |
2021-05-07 |
Fix webdispense |
[MTL] |
2021-05-06 |
Fix full /var on uccportal, add logrotate |
[BOB] |
2021-04-30 |
Fixed radiusd config on samson to suit updated requirements for full hostname and IP |
[BOB][TPG][NTU] |
2021-04-22 |
Fixed our long-broken IPSec VPN: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/2021-April/005460.html |
[333] |
2021-04-20 |
Fixed MariaDB on Mussel, and thus OCSInventory - which depends on it |
[333] |
2021-04-18 |
Completely rebuilt the idle-medico VM using only the Ansible playbook that [MTL] wrote |
[TPG] |
2021-03-08 |
Fixed gitlab certbot |
[MTL] |
2021-02-22 |
Implement age command in uccpass. Migrate uccpass repo to UCC gitlab |
[MTL] |
2021-02-20 |
Move octoDNS to dedicated venv for UCC Cloudflare DNS sync |
[MTL] |
2021-01-20 |
Poke certs for irc.ucc.asn.au |
[MPT] |
2021-01-18 |
Remove mollitz mock internal subnet from murasoi |
[MPT][NTU] |
2021-01-18 |
Tinker with mollitz config and update wiki page to match |
[TRS][MPT] |
2021-01-12 |
Return mollitz legacy backups, offsite |
[TPG] |
2021-01-11 |
molmol freebsd-update |
[NTU] |
2021-01-10 |
atlantic cluster magikarp extend ceph osd.3 |
[BOB] |
2021-01-10 |
atlantic cluster apt updates, reboots |
[TPG][NTU] |
2020-12-28,30 |
mailfish:~spamass training |
[333][NTU] |
2020-12-21 |
Replace molmol mirror SSD: root:geom-mirror boot swap zfs-slog |
[TPG] |
2020-12-21 |
webcam archiving and retrieval optimisation |
[NTU] |
2020-12-11 |
Test Ceph WAL on vmstore-bigssd, add discord-irc to legacy backups |
[NTU][MTL] |
2020-12-10 |
mailfish: Bounce due to NFS problems |
[MPT] |
2020-12-02 |
Fix Abe configuration and rancid |
[MPT] |
2020-12-02 |
Troubleshoot UniSFA TV network connection |
[MPT] |
2020-11-30 |
Reshuffle UCC wifi channels |
[NTU][MTL] |
2020-11-29 |
mussel: fix MoinMoin wiki textcha again |
[NTU][CJS][MPT] |
2020-10-19 |
place mollitz (legacy backups) on UCC network |
[MPT] |
2020-10-19 |
Create user VM for [DIE] for SS13 server |
[MTL][MPT][DAA] |
2020-10-12 |
Make login.ucc.asn.au work |
[MPT][333][NTU] |
2020-10-09--2020-10-12 |
CEPH! (Fix latency issues, lots of tweaks to Ceph config...) |
[MTL] |
2020-10-06 |
Fix gitlab.ucc.asn.au LE certs |
[MTL] |
2020-10-06 |
Fix portal.ucc.asn.au LE certs on uccportal. Clear full /tmp and /var partitions |
[MTL] |
2020-08-31 |
Fix webmail.ucc.asn.au redirect. Curse mussel apache config |
[TPG] |
2020-08-29 |
Fixed the wheel group page |
[MTL] |
2020-08-21 |
Fix wildcard certs on mailfish, mussel and motsugo via ucc-ansible-soe and Cloudflare API key |
[MTL] |
2020-07-28 |
Fix mentions of mooneye in mailfish config vis headers and spam |
[MTL] |
2020-06-08 |
Fix HTTP redirect to HTTPS for the wiki |
[MTL] |
2020-06-02 |
Add LDAP configuration for UCC AD to uccmonitor |
[MPT] |
2020-05-27 |
Enable nested KVM virtualisation on mudkip, check enabled on others |
[NTU] |
2020-05-24 |
lvextend -r murasoi:/var, add /etc/logrotate.d/ucc-ssh-auth |
[BOB] |
2020-04-25 |
dist-upgrade motsugo to buster |
[BOB] |
2020-04-25 |
Renewed letsencrypt certs on murasoi to fix ipsec vpn |
[BOB] |
2020-04-25 |
Changed main NS record with DNS registrar from mooneye.ucc.gu.uwa.edu.au to mooneye.ucc.asn.au, updated zone records to suit, and changed _acme-challenge.secure.ucc.asn.au zone to use mooneye.ucc.asn.au |
[BOB] |
2020-04-24 |
Fix rancid issues introduced by upgrade |
[BOB] |
2020-04-21 |
dist-upgrade mussel to buster |
[BOB] |
2020-04-21 |
dist-upgrade gitlab to buster |
[MTL] |
2020-04-21 |
dist-upgrade meersau to buster; rejoin AD and fix Xrdp logins - see https://github.com/neutrinolabs/xrdp/issues/906 |
[MPT] |
2020-04-18 |
L3 setup of 4G modem, enabled policy-based routing for murasoi |
[MPT] |
2020-04-17 |
Physical and L2 setup of 4G modem in machine room |
[MPT] |
2020-04-17 |
Install replacement fans and recommission mudkip |
[NTU][MTL] |
2020-04-15 |
Surgical update of mussel's DebianPkg:libapache2-mod-wsgi to 4.5.17-1_i386.deb to fix moinmoin 866180 |
[MTL] |
2020-04-13 |
Deploy FoldingAtHome to UCC clubroom machines to keep them busy during the COVID19 pandemic |
[MTL] |
2020-04-11 |
Migrate UCC wiki from mooneye to mussel |
[NTU][TRS] |
2020-04-09 |
Add mollitz (offsite backups) to monitoring |
[MTL] |
2020-04-06 |
Set up Mailauesi with haproxy for IMAPS, POP3S and SMTP submission frontend load balancer |
[TRS] |
2020-03-21 |
Fixed the VPN by pointing it at the current let's encrypt cert |
[TRS] |
2020-03-21 |
Installed BigBlueButton |
[MSH] |
2020-03-21 |
Make motsugo logrotate btmp (failed auth attempts) weekly so disk doesn't fill up |
[DAS][MPT] |
2020-02-26 |
Troubleshooting Unigames webcams, replugged loose patch cable, moved VLAN1 to new Lard trunk |
[MPT][CFE] |
2020-02-25 |
Increase 5G channel width to 40MHz into free space |
[MPT][NTU] |
2020-02-24 |
Shutdown Magikarp and diagnose fan issue |
[MSH] |
2020-02-15 |
Set mooneye:/etc/opendkim.conf "MaximumHeaders 0" to handle >64KB headers from looping mail to/from(?) [email protected] |
[MPT] |
2020-02-10 |
Convert all PVEVMUser permissions to newer UCC_VM_User, update documentation |
[BOB] |
2020-02-10 |
Audited user VM's |
[BOB] |
2020-01-28 |
Change ocsinventory to ssl only and roll out to all machines |
[MPT] |
2020-01-27 |
Resize /dev/md0 and /var on murasoi, with room to grow |
[MPT] |
2020-01-27 |
Fix uccpass regression preventing new keys being added |
[TEC][NTU][MTL] |
2020-01-13 |
Install Prometheus/Grafana monitoring, with Ansible, on uccmonitor, resize /var, add custom metric |
[TEC] |
2019-12-24 |
Install Prometheus/Grafana monitoring, with Ansible, on uccmonitor |
[MPT][NTU] |
2019-12-{16,17} |
Provision new RPi 4 as user server titan, install SOE and useful programs |
[MPT] |
2019-12-12 |
Migrate UCC mail to new IronPort cloud appliances with UWA |
[MPT][NTU] |
2019-12-09 |
Outbound mail relayhost hostname {antivirus -> smtp}.uwa.edu.au per UniIT request |
[NTU][TBB] |
2019-12-07 |
merlo /usr/local/bin/serial-connect fixme resets VTxxx terminal |
[MPT] |
2019-12-02 |
Improved network documentation somewhat |
[MPT][NTU][MTL] |
2019-12-02 |
Provisioned new "IoT devices" network, including L2, L3 and PSK wifi |
[NTU][MPT] |
2019-12-02 |
Created an AWS account for UCC |
[MPT][NTU] |
2019-11-25 |
Fix cobra networking (broken small switch), /away automounting |
[TEC][333] |
2019-11-25 |
add magikarp and mudkip to Atlantic proxmox cluster |
[TEC][MDD] |
2019-11-25 |
update minecraft2017 -> minecraft2019 |
[333] |
2019-11-21 |
Diagnose and fix smallwing's failing UniFi firmware updates |
[MPT][TPG] |
2019-11-16 |
Unborking a user account, restrict usernames to 3 or more characters |
[MPT] |
2019-11-05 |
Redirected fresher@ to rest of committee-only@ (successfully this time) |
[MSH] |
2019-10-22 |
Set kernel.dmesg_restrict=0 on motsugo and mussel |
[MPT][THA][TPG][MSH][NTU] |
2019-10-18 |
Service troubleshooting and restarts, after 0430--0521 power outage: proxmox/samba/ssd/zonemake? |
[FVP] |
2019-09-20--2019-10-11 |
username change? |
[MTL] |
2019-08-22 |
dpkg --add-architecture i386; apt update; apt install libpcre2-dev:i386 libncurses5-dev:i386 on motsugo for [DAA]. Also, free up space on full /var |
[MPT] |
2019-08-14 |
Implemented soft-blacklist for possibly-broken account names in ucc-adduser-ad |
[FVP] |
2019-08-13 |
Replaced faulty Microtik router sharpchin with spare OpenWRT device coromandel (in UniSFA) |
[MPT] |
2019-08-09 |
Changed ucc-adduser-ad to make new users' index.html readable by the webserver |
[MPT] |
2019-07-31 |
Updated MOTD to reflect [DBA]'s resignation from committee |
[MPT] |
2019-07-29 |
Removed ex-group members from their old groups on UCC systems (following group review) |
[DAA] |
2019-07-29 |
Tried and failed to upgrade Guacamole to 1.0.0 (using Postgres + RADIUS and then Postgres + LDAP), rolled back to 0.9.14 |
[333][NTU][TPG] |
2019-07-22 |
Wifi - fix smallwing/nanoHD VLAN connectivity to kerosene, was curviceps |
[MPT] |
2019-07-20 |
Enabled daily updates of Linux desktops |
[MPT] |
2019-07-20 |
Created 90-ucc-desktops.conf for PolKit and deployed to all desktops, enabled unattended upgrades on all desktops |
[MPT] |
2019-07-18 |
Joined Catfish's Linux install to the domain, fixed network logins |
[TPG][NTU] |
2019-07-15 |
Switch complications? |
[TRS][NTU] |
2019-07-14 |
UWA DNS outage, check bind+dd/discord-irc, disable/enable mooneye DNS forwarders |
[FVP][MPT][DAS] |
2019-07-12 |
Replaced machine room switch with new chassis and supervisor with 10GE capability |
[DAA] |
2019-07-11 |
Enlarged /backups array, partition, logical volume and filesystem on Mollitz |
[DAA] |
2019-07-09 |
Installed FreeBSD updates to 11.2-RELEASE-p11 on Molmol & upgraded pkg set |
[MPT] |
2019-06-28 |
Reinstalled clownfish with updated Mint 19.1 on the new NVMe SSD |
[BOB] |
2019-06-25 |
Changed door mailing list settings to discard spam emails |
[BOB][MPT][FVP][AJT][TRS] |
2019-06-21--2019-06-22 |
Recovered murasoi from dual system disk failure |
[MSH] |
2019-06-18 |
Filtered SACK on murasoi ucc-fw to avoid SACK Panic |
[MPT] |
2019-06-10 |
Fixed /etc/fstab on pinball to boot new NVMe SSD |
[MPT] |
2019-06-10 |
Swapped in new fans to pinball and clownfish (with [AJT]) |
[MPT] |
2019-06-09 |
Fixed network issues on upgraded clownfish |
[AJT] |
2019-06-07 |
Upgraded pinball, clownfish and porcupine with new components (with [MPT]) |
[DAA] |
2019-05-27 |
Upgraded Samba on Samson from 4.9.2 to 4.9.5 |
[MTL] |
2019-05-20 |
Some housekeeping done on mooneye /etc/aliases. All mailman mailing lists moved to bottom of file in prep for later mailman upgrade |
[DAA] |
2019-05-18 |
Removed a stack of obsolete Python packages that were breaking things |
[DAA] |
2019-05-18 |
Gitlab maintenance - fixed UCC user accounts to be internal, not external, and removed heaps of spam accounts |
[DAA] |
2019-05-15 |
Converted Apache on Mussel to Certbot and wildcard SSL; removed the acme-members scripts from crontab and zonemake; converted Dovecot on Motsugo to Certbot and wildcard SSL; converted IPsec on Murasoi to use DNS challenges; dropped acmetool on Mussel, Mooneye, Motsugo, Murasoi and Gitlab; wrote SSLCertificates page |
[DAA] |
2019-05-15 |
Security patches installed for Molmol, Meersau and Maaxen |
[DAA] |
2019-05-13 |
Converted inspircd on Mussel to Certbot |
[DAA] |
2019-05-13 |
Converted postfix and apache on Mooneye to Certbot and wildcard SSL |
[DAA] |
2019-05-12 |
|
[BOB][DBA] |
2019-05-03 |
Installed replacement system/slog disk SSD in molmol |
[DAA] |
2019-05-03 |
Removed a large number of obsolete packages (mostly libraries and old development tools) from Mussel |
[DAA] |
2019-05-02 |
Upgraded Molmol to FreeBSD 11.2-RELEASE |
[FVP] |
2019-05-01 |
Refactored uccpass to put user keys in separate files, with committee and wheel directories |
[MPT] |
2019-04-30 |
Removed elliotnunn from local sudo /etc/group on clownfish to stop Polkit from asking everyone for his password |
[MPT] |
2019-04-30 |
Added [MLG] to committee user group, and updated motd |
[DAA] |
2019-04-29 |
Upgraded to Debian 9.9 on Mussel, Motsugo, Murasoi, Mooneye, Merlo, Samson, Mollitz, Gitlab, idle-Maltair, idle-Medico, Heathred, Medico, Maltair and Loveday; ceph restarted and checked for status |
[BOB] |
2019-04-27 |
Updated salmon and installed unattended upgrades. Sorted through VM cluster and tagged a heap of VM's for deletion |
[BOB] |
2019-04-26 |
|
[MSH] |
2019-04-23 |
Made DNS notifies/AXFR from secondaries work |
[MSH] |
2019-04-23 |
|
[DAA] |
2019-04-20 |
Removed a bunch of obsolete packages from motsugo (gcc-4.6 ruby1.8 ruby1.9.1 php5-cli postgresql-client-9.4 python2.6-minimal zlib-bin libjasper1 hardening-includes python-reportbug python-support sysvinit) |
[BEN] |
2019-04-18 |
Setup unisfa-freshers mailing list |
[NTU][TEC] |
2019-04-15 |
ocsinventory-agent deployed to corvo, napoli |
[BOB] |
2019-04-13 |
Set up ocsinventory.ucc.asn.au and deployed to most machines |
[DAA] |
2019-04-12 |
Upgraded idle-{medico,maltair} to Debian 9; installed Docker and Gitlab Runner on idle-medico |
[DAA] |
2019-04-10 |
Installed rust and cargo from testing on Motsugo |
[SJH][MSH] |
2019-04-09 |
Fixed user php7 |
[DAA] |
2019-04-09 |
|
[BOB] |
2019-04-09 |
|
[DAA] |
2019-04-09 |
Attempted to fix eggman mounting /mp3s on boot by enabling systemd-networkd-wait-online.service |
[DAA] |
2019-04-09 |
Removed motsugo pip packages for Pygments, Six, and youtube-dl as they were old and newer Debian packages were installed |
[MTL] |
2019-04-03 |
mussel package upgrades |
[MSH] |
2019-04-02 |
Removed and readded motsugo sdc and sde. Got rid of current-pending-sectors by overwriting them |
[NTU] [MPT] |
2019-04-01 |
Merge merlo:/var/log/dispense into ~coke/cokelog , restart rsyslogd on merlo, motsugo |
[MPT] |
2019-03-27 |
Remounted /home and /away on merlo to fix BAD PIN message |
[BOB] |
2019-03-26 |
Re-synced time on Loveday - clock drift was causing ceph to be unhealthy |
[BOB] |
2019-03-26 |
Installed LSI SAS card in new HP DL380 (maltair), updated assorted firmwares, fixed network interfaces and re-added to VM cluster |
[DTA] |
2019-03-26 |
Set passwords for flame users id, dos, linux - it might stop the problem with aserver.err filling up after a portscan |
[MSH] |
2019-03-23 |
put semrushbot in secure.ucc, wiki.ucc robots.txt (not sure if wiki worked |
[NTU] [MPT] |
2019-03-18 |
reset napoli passwd, add sudoers |
[MSH] |
2019-03-18 |
Got arpwatch running again on murasoi, had to put br0.* in /etc/arpwatch.conf |
[NTU] [TPG] |
2019-03-16 |
merlo not remote logging and BAD PIN? reboot, remount home, restart rsyslogd on merlo, motsugo |
[NTU] [FVP] [JGM] |
2019-03-16 |
Server rack in corridor, temporary? switch to VT320 with a narrower keyboard until a narrow PS/2 keyboard is found for the VT510 https://lists.ucc.gu.uwa.edu.au/pipermail/tech/2019-March/005091.html |
[MSH] |
2019-03-16 |
Created mooneye:/var/log/journal so logs will persist across reboots |
[RME] |
2019-03-15 |
Created this page. Let's see if it sticks. |
[BOB] |
2019-03-14 |
Removed root keys of wheel members who are not current members |
[BOB] |
2019-03-05 |
Attended Ross's auctions to inspect HP DL380 Gen 8 server for maltair replacement. Bid and won in auction. Delivered to UCC and installed |
[TRS] |
2019-02-05 |
Pointed SOGo to AD (was still LDAP.) |
[TRS] |
2018-12-01 |
Upgraded SOGo on mussel to 4.0.4. |
[FVP] [CFE] |
2018-11 to 2019-02 |
Created portal.ucc.asn.au based on [DAA]'s MemberDB for membership information |