|
Size: 3767
Comment:
|
Size: 7907
Comment: UWAES / E-sports network VLAN 209
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| Understanding UCC's network can be a bit challenging at first, but after a bit of reading you'll find that it is actually very challenging, and give up. This article is a general overview of how it works. [[TableOfContents]] |
Understanding UCC's network can be a bit challenging at first, but after some reading you'll find that it is actually very challenging, and give up. This article is a general overview of how it works. <<TableOfContents>> |
| Line 5: | Line 5: |
| There is a long piece of CAT5 running through the walls from the machine room to the Guild machine room in Cameron Hall (across from UWAnime). This plugs in to a 100M D-Link media converter, which leads to a similar media converter in the Guild comms room in the main Guild building. Our uplink is into an ITS managed switch called 'cruzob'. If your're looking for where the cable runs, it's possibly disguised as a network outlet cable in one of the other student clubrooms. | There is a [[https://en.wikipedia.org/wiki/Category_6_cable|Cat 6]] cable running straight up out of the Guild machine room, along the rafters and back down into the UCC machine room, terminated on a block on the North machine room wall. <<BR>>`murasoi:eth1 <-patch-> Gi2/2 kerosene Gi2/19 <---cat6---> Gi7/1 lard` In addition, there is a long piece of [[https://en.wikipedia.org/wiki/Category_5_cable|Cat 5]] (that was previously the primary uplink!) running through the walls from the machine room to the Guild machine room in Cameron Hall (across from Newnigames). If you're looking for where the cable runs, it may look like a network outlet cable in one of the other student clubrooms. <<BR>>`murasoi:br0:eth2 <-fibre patch 10GBASE-SR-> 0/1 walnut 0/7 <-fibre patch 10GBASE-SR-> Te1/1 kerosene Gi2/1 <---cat5---> Gi7/2 lard` In the Guild machine room is a Cisco 4507R --(484 watt media converter)-- switch called [[Network/SwitchConfiguration#Lard|Lard]]. This connects to the Cat5/Cat6 above and to single mode fibre, which runs into an ITS managed distribution switch located in the Science library and is imaginatively titled [[http://netmap.ucs.uwa.edu.au/netmap/index.cgi?devip=10.10.3.7|science-dr-01]]. <<BR>>`lard <----SMF-1000BASE-LX----> science-dr-01 <----SMF-10GBASE-LR?----> UWA-IT <-----SMF-100GBASE-LR?-----> AARNet,world` |
| Line 7: | Line 15: |
| The machine rooms contains three core switches and a router: * Olive, a 24-port Cisco Catalyst 2900 series switch. * Lorenzo, a 48-port Cisco Catalyst 2950 series with some dead ports and dual gigabit uplinks. * Curviceps, a 24-port HP Procurve with full gig ports. * Madako, a linux-based router running iptables. These are all labeled and in the rack. There is also a patch panel for the clubroom wall-ports at the top of the rack. |
The machine room contains three switches and a router: * [[Murasoi]], a GNU/Linux-based router running iptables/nftables. * [[Network/SwitchConfiguration#Walnut|Walnut]], a Ubiquiti EdgeSwitch 16-XG, connecting the router, other switches, and 10Gbps servers * [[Network/SwitchConfiguration#Kerosene|Kerosene]], a Cisco Catalyst 4506-E running IOS which has a Supervisor 6-E engine, 96 GigE ports and 2 10GE-capable X2 slots. * [[Network/SwitchConfiguration#Curviceps|Curviceps]], an HP ProCurve 1800-24G which has 24 GigE ports and 2 SFP slots. These are all labeled and in the rack. There is also a patch panel for the clubroom wall-ports at the bottom of the right-most rack beneath kerosene. |
| Line 14: | Line 22: |
| There is CAT5 cabling run from a patch panel at the top of the rack to a number of wall ports throughout the room. Where not enough wall-ports are available, there are small 5-port unmanaged switches used to attach more devices to the network. | There is Cat5 cabling run from a patch panel at the bottom of the rack to a number of wall ports throughout the room. Where not enough wall ports are available, there are small 5-port unmanaged switches used to attach more devices to the network. The [[Wifi|wireless network]] is also available in the clubroom. |
| Line 18: | Line 29: |
| * VLAN 1: Network and server management. * VLAN 2: Machine room network. * VLAN 3: Clubroom network. * VLAN 5: Loft network (used for LANs). * VLAN 6: Wireless network. * VLAN 7: Printers. * VLAN 8: Netboot (Ubuntu port) |
* VLAN 1: Network and server management * VLAN 2: Machine room network * VLAN 3: Clubroom network * VLAN 4: Member VM network * VLAN 5: Loft network (used for LANs) * VLAN 6: Authenticated wireless network * VLAN 7: IoT device network (wired & wireless) * --(VLAN 8: Untrusted wireless network.)-- (deprecated) * VLAN 209: UWAES / E-sports network (wired & wireless) * VLAN 999: Used as a dummy/blackhole VLAN on some switches. Do not trunk |
| Line 26: | Line 40: |
| ITS trunks to us the following VLANs: * VLAN 11: SNAP. * VLAN 13: Our main uplink, provides us our internet connection and address space. * VLAN 102: Guild clubs. Not used by UCC, forwarded on to UniSFA. |
University IT trunks the following VLAN to us: * VLAN 13: Our main uplink, provides us our internet connection and address space |
| Line 31: | Line 43: |
| Layer three at UCC is pretty nasty, and the firewall script alone probable deserves its own article. However, a brief summary of how it all works: | Layer three at UCC is reasonably straightforward these days. A brief summary: |
| Line 34: | Line 46: |
| * 130.95.13.0/24 is the public address space for our AARNet connection. Incoming, non-peering traffic to these addresses is charged at 4c/mb. This range is routed to us via VLAN 13. | * 130.95.13.0/24 is the public address space for our AARNet connection. This range is routed to us via VLAN 13. |
| Line 37: | Line 49: |
| * 203.24.97.249/29 is the public address space for our Silk connection. Traffic to and from these addresses is unmetered. This range is also routed to us via VLAN 13. * 10.13.13.0/24 is a private range used for network printers. These addresses reside on VLAN 7 and are not routed outside. * 10.203.13.0/24 is our address range on the Resnet (college) network. Routed via VLAN 13. * 172.26.42.0/24 is for 'untrusted client machines' - there is some history here, but they are never routed outside the Uni (unless NAT is involved, which it is). * 172.26.42.8/29 is the Ubuntu Port (netboot install) network. * 172.26.42.96/27 is the range we use for PPTP. * 172.26.42.128/26 is the loft network range. * 172.26.42.192/27 is the 'ugg' wireless network range * 172.26.24.224/27 is used by Flying for the 'ucc' wireless network, currently unavailable. * 10.11.0.0/24 on the SNAP vlan (10.11.0.0/16 in total) is not leased out by the central SNAP dhcp server, so some IPs in this range can be taken for services. Currently 10.11.0.13 is the UCC SNAP SSH forward (which ends up at martello) and .11 is used by evil. |
* 130.95.13.128/26 is the member VM address range, internally routed on VLAN 4. * 192.168.2.0/24 is the management VLAN IP range. This is not allocated to us by UWA and not routed outside UCC. * 192.168.4.0/24 is the private subnet carrying traffic between Murasoi and the 4G modem. * 192.168.5.0/24 is the subnet for the "4G backup" Wireguard tunnel between UCC and AWS. * 192.168.9.0/24 is the Virtual UCC (VUCC) network (ask [FVP]), routable via `vucc0.ucc.asn.au` (`130.95.13.35`) * 192.168.11.0/24 is the subnet for the "proxy" Wireguard links between UCC and BinaryLane. * 192.168.13.0/24 is allocated to us by UWA and routed on VLAN 13. * 192.168.13.0/25 is the uplink network from Murasoi to UWA. * 192.168.13.128/25 is the former range for machines colocated in Arts. * Currently unused. * 192.168.16.0/22 is the authenticated UCC clients range * 192.168.16.0/24 is the [[Wifi|UCC wifi]] range * 192.168.18.0/24 is the IPsec VPN client range * 192.168.19.0/24 is the OpenVPN client range * 192.168.20.0/22 is the untrusted / unauthenticated UCC range * 192.168.20.0/24 is the ''new'' Loft range * 192.168.22.0/24 is the IoT range * 172.26.42.0/24 is for 'untrusted client machines' and is allocated to us by UWA and routed to us via VLAN 13. There is some history here, but these addresses are not routed outside the Uni. This subnet may be NATted to public IPs for external access. * Currently unused. === Addressing scheme === Most of UCC's subnets use DHCP to assign addresses based on MAC address. Further details can be found at [[Network/Services#DHCP]]. === Routing and Firewall === [[Murasoi]], the GNU/Linux router, is a beast of burden. See [[Network/Firewall]] for further information on Layer 3 routing and firewalling configuration. === IPv6 === [[http://ipv6.he.net/certification/scoresheet.php?pass_name=accmurphy|{{http://ipv6.he.net/certification/make_badge.php?pass_name=accmurphy}}]] UCC has 2405:3C00:5200:100::/58 (which is :100:: to :13f:: inclusive; in other words, 64 networks). This is an unusually small CIDR block. [[http://tools.ietf.org/html/rfc6177|RFC-6177]] recommends that small end sites - such as a home user with devices in the "dozens or less" - should be allocated a /56 block (256 networks). This is advertised by radvd on [[Murasoi]] which most machines autoconfigure from, however some machines have statically assigned addresses. There is an IPv6 firewall that matches our IPv4 firewall very closely. Many machine room systems have IPv6 address, which are statically assigned. There is no reverse DNS delegation at this stage, so `...ip6.arpa.` reverse DNS is UCC-only. [[Monnik]]'s DNS record doesn't have an AAAA record, because we are scared of this breaking Things(tm). IPv6 is routed to 2405:3C00:10:4::1 from [[Murasoi]]. ==== Subnets ==== * 2405:3C00:10:4::2/64 uplink (equivalent to 192.168.13.2, UWA VLAN 13) * 2405:3C00:5200:100::/64 machine room (VLAN 2) * 2405:3C00:5200:101::/64 clubroom (VLAN 3) * 2405:3C00:5200:102::/64 member VMs (VLAN 4) * 2405:3C00:5200:103::/64 loft (VLAN 5) * 2405:3C00:5200:104::/64 wireless (VLAN 6) * 2405:3C00:5200:105::/64 IoT (VLAN 7) * --(2405:3C00:5200:106::/64 public wireless (VLAN 8))-- * 2405:3C00:5200:120::/120 IPsec VPN * 2405:3c00:5200:121::/64 OpenVPN * 2405:3c00:5200:9100::/64 VUCC "Virtual UCC" IP range (note: not technically owned by UCC and hence not routable from the Internet) === Multicast === UWA runs multicast in sparse PIM mode, and [[Murasoi]] runs pimd as noted [[http://lists.ucc.gu.uwa.edu.au/pipermail/ucc/2006-October/013668.html|here]]. Make sure pimd is only listening once per interface, otherwise things won't work quite right. == Higher Layers == Higher layer configuration at UCC has changed significantly since TheCloudflarening. See that new page for more details. == Monitoring == There are various monitoring packages installed, links to which can be found on MissionControl. ---- CategorySystemAdministration |
Understanding UCC's network can be a bit challenging at first, but after some reading you'll find that it is actually very challenging, and give up. This article is a general overview of how it works.
Contents
Layer One
Uplink
There is a Cat 6 cable running straight up out of the Guild machine room, along the rafters and back down into the UCC machine room, terminated on a block on the North machine room wall.
murasoi:eth1 <-patch-> Gi2/2 kerosene Gi2/19 <---cat6---> Gi7/1 lard
In addition, there is a long piece of Cat 5 (that was previously the primary uplink!) running through the walls from the machine room to the Guild machine room in Cameron Hall (across from Newnigames). If you're looking for where the cable runs, it may look like a network outlet cable in one of the other student clubrooms.
murasoi:br0:eth2 <-fibre patch 10GBASE-SR-> 0/1 walnut 0/7 <-fibre patch 10GBASE-SR-> Te1/1 kerosene Gi2/1 <---cat5---> Gi7/2 lard
In the Guild machine room is a Cisco 4507R 484 watt media converter switch called Lard. This connects to the Cat5/Cat6 above and to single mode fibre, which runs into an ITS managed distribution switch located in the Science library and is imaginatively titled science-dr-01.
lard <----SMF-1000BASE-LX----> science-dr-01 <----SMF-10GBASE-LR?----> UWA-IT <-----SMF-100GBASE-LR?-----> AARNet,world
Machine Room
The machine room contains three switches and a router:
Murasoi, a GNU/Linux-based router running iptables/nftables.
Walnut, a Ubiquiti EdgeSwitch 16-XG, connecting the router, other switches, and 10Gbps servers
Kerosene, a Cisco Catalyst 4506-E running IOS which has a Supervisor 6-E engine, 96 GigE ports and 2 10GE-capable X2 slots.
Curviceps, an HP ProCurve 1800-24G which has 24 GigE ports and 2 SFP slots.
These are all labeled and in the rack. There is also a patch panel for the clubroom wall-ports at the bottom of the right-most rack beneath kerosene.
Clubroom
There is Cat5 cabling run from a patch panel at the bottom of the rack to a number of wall ports throughout the room. Where not enough wall ports are available, there are small 5-port unmanaged switches used to attach more devices to the network.
The wireless network is also available in the clubroom.
Layer Two
Internal VLANs
UCC uses seven VLANs internally for various purposes:
- VLAN 1: Network and server management
- VLAN 2: Machine room network
- VLAN 3: Clubroom network
- VLAN 4: Member VM network
- VLAN 5: Loft network (used for LANs)
- VLAN 6: Authenticated wireless network
VLAN 7: IoT device network (wired & wireless)
VLAN 8: Untrusted wireless network. (deprecated)
VLAN 209: UWAES / E-sports network (wired & wireless)
- VLAN 999: Used as a dummy/blackhole VLAN on some switches. Do not trunk
External VLANs
University IT trunks the following VLAN to us:
- VLAN 13: Our main uplink, provides us our internet connection and address space
Layer Three
Layer three at UCC is reasonably straightforward these days. A brief summary:
Subnets
There are a number of IP ranges used at UCC for various things:
- 130.95.13.0/24 is the public address space for our AARNet connection. This range is routed to us via VLAN 13.
- 130.95.13.0/26 is the machine room address range, internally routed on VLAN 2.
- 130.95.13.64/26 is the clubroom address range, internally routed on VLAN 3.
- 130.95.13.128/26 is the member VM address range, internally routed on VLAN 4.
- 192.168.2.0/24 is the management VLAN IP range. This is not allocated to us by UWA and not routed outside UCC.
- 192.168.4.0/24 is the private subnet carrying traffic between Murasoi and the 4G modem.
- 192.168.5.0/24 is the subnet for the "4G backup" Wireguard tunnel between UCC and AWS.
192.168.9.0/24 is the Virtual UCC (VUCC) network (ask [FVP]), routable via vucc0.ucc.asn.au (130.95.13.35)
192.168.11.0/24 is the subnet for the "proxy" Wireguard links between UCC and BinaryLane.
- 192.168.13.0/24 is allocated to us by UWA and routed on VLAN 13.
- 192.168.13.0/25 is the uplink network from Murasoi to UWA.
- 192.168.13.128/25 is the former range for machines colocated in Arts.
- Currently unused.
- 192.168.16.0/22 is the authenticated UCC clients range
192.168.16.0/24 is the UCC wifi range
- 192.168.18.0/24 is the IPsec VPN client range
- 192.168.19.0/24 is the OpenVPN client range
- 192.168.20.0/22 is the untrusted / unauthenticated UCC range
192.168.20.0/24 is the new Loft range
- 192.168.22.0/24 is the IoT range
- 172.26.42.0/24 is for 'untrusted client machines' and is allocated to us by UWA and routed to us via VLAN 13. There is some history here, but these addresses are not routed outside the Uni. This subnet may be NATted to public IPs for external access.
- Currently unused.
Addressing scheme
Most of UCC's subnets use DHCP to assign addresses based on MAC address. Further details can be found at Network/Services#DHCP.
Routing and Firewall
Murasoi, the GNU/Linux router, is a beast of burden. See Network/Firewall for further information on Layer 3 routing and firewalling configuration.
IPv6
UCC has 2405:3C00:5200:100::/58 (which is :100:: to :13f:: inclusive; in other words, 64 networks).
This is an unusually small CIDR block. RFC-6177 recommends that small end sites - such as a home user with devices in the "dozens or less" - should be allocated a /56 block (256 networks).
This is advertised by radvd on Murasoi which most machines autoconfigure from, however some machines have statically assigned addresses. There is an IPv6 firewall that matches our IPv4 firewall very closely.
Many machine room systems have IPv6 address, which are statically assigned. There is no reverse DNS delegation at this stage, so ...ip6.arpa. reverse DNS is UCC-only.
Monnik's DNS record doesn't have an AAAA record, because we are scared of this breaking Things(tm).
IPv6 is routed to 2405:3C00:10:4::1 from Murasoi.
Subnets
- 2405:3C00:10:4::2/64 uplink (equivalent to 192.168.13.2, UWA VLAN 13)
- 2405:3C00:5200:100::/64 machine room (VLAN 2)
- 2405:3C00:5200:101::/64 clubroom (VLAN 3)
- 2405:3C00:5200:102::/64 member VMs (VLAN 4)
- 2405:3C00:5200:103::/64 loft (VLAN 5)
- 2405:3C00:5200:104::/64 wireless (VLAN 6)
- 2405:3C00:5200:105::/64 IoT (VLAN 7)
2405:3C00:5200:106::/64 public wireless (VLAN 8)
- 2405:3C00:5200:120::/120 IPsec VPN
- 2405:3c00:5200:121::/64 OpenVPN
- 2405:3c00:5200:9100::/64 VUCC "Virtual UCC" IP range (note: not technically owned by UCC and hence not routable from the Internet)
Multicast
UWA runs multicast in sparse PIM mode, and Murasoi runs pimd as noted here. Make sure pimd is only listening once per interface, otherwise things won't work quite right.
Higher Layers
Higher layer configuration at UCC has changed significantly since TheCloudflarening. See that new page for more details.
Monitoring
There are various monitoring packages installed, links to which can be found on MissionControl.
