Differences between revisions 68 and 89 (spanning 21 versions)
⇤ ← Revision 68 as of 2018-06-12 12:08:38 →
Size: 7306
Editor: frekk
Comment:
← Revision 89 as of 2024-10-14 20:00:05 → ⇥
Size: 7964
Editor: 2405:3c00:5200:104:8a3d:73e0:f141:efb9
Comment: docker
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
There is a CAT6 cable running straight up out of the Guild machine room, along the rafters and back down into the UCC machine room, terminated on a block on the North machine room wall. There is a [[https://en.wikipedia.org/wiki/Category_6_cable|Cat 6]] cable running straight up out of the Guild machine room, along the rafters and back down into the UCC machine room, terminated on a block on the North machine room wall.
<<BR>>`murasoi:eth1 <-patch-> Gi2/2 kerosene Gi2/19 <---cat6---> Gi7/1 lard`
Line 7: Line 8:
In addition, there is a long piece of CAT5 (that was previously the primary uplink!) running through the walls from the machine room to the Guild machine room in Cameron Hall (across from UWAnime). If you're looking for where the cable runs, it's possibly disguised as a network outlet cable in one of the other student clubrooms. In addition, there is a long piece of [[https://en.wikipedia.org/wiki/Category_5_cable|Cat 5]] (that was previously the primary uplink!) running through the walls from the machine room to the Guild machine room in Cameron Hall (across from Newnigames). If you're looking for where the cable runs, it may look like a network outlet cable in one of the other student clubrooms.
<<BR>>`murasoi:br0:eth2 <-fibre patch 10GBASE-SR-> 0/1 walnut 0/7 <-fibre patch 10GBASE-SR-> Te1/1 kerosene Gi2/1 <---cat5---> Gi7/2 lard`
Line 9: Line 11:
In the Guild machine room is a --(Cisco 3508XL called `sesame`)-- Cisco 4507 called `lard` but in DNS as `sesame`. This connects to the CAT5/CAT6 above and to single mode fibre, which runs into an ITS managed distribution switch located in the Science library and is imaginatively titled [[http://netmap.ucs.uwa.edu.au/netmap/index.cgi?devip=10.10.3.7|science-dr-01]]. In the Guild machine room is a Cisco 4507R --(484 watt media converter)-- switch called [[Network/SwitchConfiguration#Lard|Lard]]. This connects to the Cat5/Cat6 above and to single mode fibre, which runs into an ITS managed distribution switch located in the Science library and is imaginatively titled [[http://netmap.ucs.uwa.edu.au/netmap/index.cgi?devip=10.10.3.7|science-dr-01]].
<<BR>>`lard <----SMF-1000BASE-LX----> science-dr-01 <----SMF-10GBASE-LR?----> UWA-IT <-----SMF-100GBASE-LR?-----> AARNet,world`
Line 12: Line 15:
The machine room contains four switches and a router:
 * [[Murasoi]], a linux-based router running iptables.
 * [[Bitumen]], a Cisco Catalyst 4507R running IOS which has 2 SupIV supervisor engines, 96 GigE ports and 12 GBIC slots.
 * [[Curviceps]], an HP ProCurve 1800-24G which has 24 GigE ports and 2 SFP slots.
 * [[Walnut]], a Ubiquiti EdgeSwitch 16-XG which has 12 SFP+ slots and 4 GigE ports.
These are all labeled and in the rack. There is also a patch panel for the clubroom wall-ports at the bottom of the right-most rack beneath Bitumen.		 The machine room contains three switches and a router:
 * [[Murasoi]], a GNU/Linux-based router running iptables/nftables.
 * [[Network/SwitchConfiguration#Walnut|Walnut]], a Ubiquiti EdgeSwitch 16-XG, connecting the router, other switches, and 10Gbps servers
 * [[Network/SwitchConfiguration#Kerosene|Kerosene]], a Cisco Catalyst 4506-E running IOS which has a Supervisor 6-E engine, 96 GigE ports and 2 10GE-capable X2 slots.
 * [[Network/SwitchConfiguration#Curviceps|Curviceps]], an HP ProCurve 1800-24G which has 24 GigE ports and 2 SFP slots.
These are all labeled and in the rack. There is also a patch panel for the clubroom wall-ports at the bottom of the right-most rack beneath kerosene.
Line 19: Line 22:
There is CAT5 cabling run from a patch panel at the bottom of the rack to a number of wall ports throughout the room. Where not enough wall-ports are available, there are small 5-port unmanaged switches used to attach more devices to the network. There is Cat5 cabling run from a patch panel at the bottom of the rack to a number of wall ports throughout the room. Where not enough wall ports are available, there are small 5-port unmanaged switches used to attach more devices to the network.
Line 24: Line 27:
See also: [[Network/SwitchConfiguration]]
Line 27: Line 29:
 * VLAN 1: Network and server management.
 * VLAN 2: Machine room network.
 * VLAN 3: Clubroom network.		  * VLAN 1: Network and server management
 * VLAN 2: Machine room network
 * VLAN 3: Clubroom network
Line 31: Line 33:
 * VLAN 5: Loft network (used for LANs).
 * VLAN 6: Authenticated wireless network.		  * VLAN 5: Loft network (used for LANs)
 * VLAN 6: Authenticated wireless network
 * VLAN 7: IoT device network (wired & wireless)
Line 34: Line 37:
 * VLAN 209: UWAES / E-sports network (wired & wireless)
 * VLAN 999: Used as a dummy/blackhole VLAN on some switches. Do not trunk
Line 35: Line 40:
ITS trunks to us the following VLAN:
 * VLAN 13: Our main uplink, provides us our internet connection and address space.		 University IT trunks the following VLAN to us:
 * VLAN 13: Our main uplink, provides us our internet connection and address space
Line 46: Line 51:
 * 192.168.13.0/25 is the uplink range, routed on VLAN 13 by UWA. Machines colocated in Arts have addresses in the top half of this /24 (i.e. 192.168.13.128/25).  * 192.168.4.0/24 is the private subnet carrying traffic between Murasoi and the 4G modem.
 * 192.168.5.0/24 is the subnet for the "4G backup" Wireguard tunnel between UCC and AWS.
 * 192.168.9.0/24 is the Virtual UCC (VUCC) network (ask [FVP]), routable via `vucc0.ucc.asn.au` (`130.95.13.35`)
 * 192.168.11.0/24 is the subnet for the "proxy" Wireguard links between UCC and BinaryLane.
 * 192.168.13.0/24 is allocated to us by UWA and routed on VLAN 13.
  * 192.168.13.0/25 is the uplink network from Murasoi to UWA.
  * 192.168.13.128/25 is the former range for machines colocated in Arts.
   * Currently unused.
Line 48: Line 60:
  * 192.168.16.0/23 is the [[Wifi|UCC wifi]] range (authenticated and accounted).
  * 192.168.18.0/24 is the IPsec VPN client range (authenticated and accounted)
  * 192.168.19.0/24 is the OpenVPN client range (authenticated and accounted)		   * 192.168.16.0/24 is the [[Wifi|UCC wifi]] range
  * 192.168.18.0/24 is the IPsec VPN client range
  * 192.168.19.0/24 is the OpenVPN client range
Line 53: Line 65:
  * 192.168.22.0/24 is the IoT range
Line 55: Line 68:
 * 172.31.0.0/16 is for Docker bridge networks on hosts
Line 59: Line 73:
[[Murasoi]], the Linux router, is a beast of burden. See [[Network/Firewall]] for further information on the way it operates. [[Murasoi]], the GNU/Linux router, is a beast of burden. See [[Network/Firewall]] for further information on Layer 3 routing and firewalling configuration.
Line 64: Line 78:
UCC has 2405:3C00:5200:100::/58 (which is :100:: to :13f:: inclusive). UCC has 2405:3C00:5200:100::/58 (which is :100:: to :13f:: inclusive; in other words, 64 networks).
Line 66: Line 80:
(This is unusually small. [[http://tools.ietf.org/html/rfc6177|RFC-6177]] recommends that small end sites - such as a home user with devices in the "dozens or less" - should be allocated a /56 range.) This is an unusually small CIDR block. [[http://tools.ietf.org/html/rfc6177|RFC-6177]] recommends that small end sites - such as a home user with devices in the "dozens or less" - should be allocated a /56 block (256 networks).
Line 68: Line 82:
This is advertised by radvd on [[Murasoi]] which most machines autoconfigure from, however some machines have statically assigned addresses. There is an IPv6 firewall that matches our IPv4 firewall very closely. IPv6 traffic is free. This is advertised by radvd on [[Murasoi]] which most machines autoconfigure from, however some machines have statically assigned addresses. There is an IPv6 firewall that matches our IPv4 firewall very closely.
Line 70: Line 84:
Many machine room systems have IPv6 address, which are statically assigned. These are available in DNS using the ipv6.ucc zone (e.g. martello.ipv6.ucc.asn.au), and usually in the main DNS entry. There is no reverse DNS delegation at this stage, so reverse DNS is UCC-only. Many machine room systems have IPv6 address, which are statically assigned. There is no reverse DNS delegation at this stage, so `...ip6.arpa.` reverse DNS is UCC-only.
Line 72: Line 86:
Mooneye's DNS record doesn't have an AAAA record, because we are scared of this breaking Things(tm). [[Monnik]]'s DNS record doesn't have an AAAA record, because we are scared of this breaking Things(tm).
Line 75: Line 89:

Mail will be received over IPv6 if it is sent to [email protected] (or ipv6.ucc.gu.uwa.edu.au).

Subnets:		 ==== Subnets ====
Line 85: Line 96:
 * 2405:3C00:5200:105::/64 IoT (VLAN 7)
Line 86: Line 98:
 * 2405:3C00:5200:120::/123 IPsec VPN (only 30 addresses to match the IPv4 pool)  * 2405:3C00:5200:120::/120 IPsec VPN
 * 2405:3c00:5200:121::/64 OpenVPN
 * 2405:3c00:5200:9100::/64 VUCC "Virtual UCC" IP range (note: not technically owned by UCC and hence not routable from the Internet)
Line 89: Line 103:
UWA runs multicast in sparse PIM mode, and [[Murasoi]] runs pimd as noted [[http://lists.ucc.gu.uwa.edu.au/pipermail/ucc/2006-October/013668.html|here]]. Make sure pimd is only listening once per interface, otherwise things won't work quite right. Multicast traffic is also free. UWA runs multicast in sparse PIM mode, and [[Murasoi]] runs pimd as noted [[http://lists.ucc.gu.uwa.edu.au/pipermail/ucc/2006-October/013668.html|here]]. Make sure pimd is only listening once per interface, otherwise things won't work quite right.
Line 92: Line 106:
HTTP goes through mussel or mooneye. HTTPS is served by mussel on secure.ucc.asn.au, however IMAPS, POPS and SMTPS are NATted by [[Murasoi]] to go to [[Motsugo]], for the historical reason that UCC could only afford one SSL certificate. Nowadays, UCC uses LetsEncrypt for everything (including secure.ucc.asn.au) and this is no longer necessary. Higher layer configuration at UCC has changed significantly since TheCloudflarening. See that new page for more details.
Line 94: Line 108:
There are a number of VPN servers running on [[Murasoi]], though SSH tends to be the most reliable protocol for tunneling about UWA.
== Configuration ==
Information on configuring the core switches can be found at [[Network/SwitchConfiguration]]. Information on configuring routing and firewalling can be found at [[Network/Firewall]].

Understanding UCC's network can be a bit challenging at first, but after some reading you'll find that it is actually very challenging, and give up. This article is a general overview of how it works.

Contents

  1. Layer One
    1. Uplink
    2. Machine Room
    3. Clubroom
  2. Layer Two
    1. Internal VLANs
    2. External VLANs
  3. Layer Three
    1. Subnets
    2. Addressing scheme
    3. Routing and Firewall
    4. IPv6
      1. Subnets
    5. Multicast
  4. Higher Layers
  5. Monitoring

Layer One

There is a Cat 6 cable running straight up out of the Guild machine room, along the rafters and back down into the UCC machine room, terminated on a block on the North machine room wall.
murasoi:eth1 <-patch-> Gi2/2 kerosene Gi2/19 <---cat6---> Gi7/1 lard

In addition, there is a long piece of Cat 5 (that was previously the primary uplink!) running through the walls from the machine room to the Guild machine room in Cameron Hall (across from Newnigames). If you're looking for where the cable runs, it may look like a network outlet cable in one of the other student clubrooms.
murasoi:br0:eth2 <-fibre patch 10GBASE-SR-> 0/1 walnut 0/7 <-fibre patch 10GBASE-SR-> Te1/1 kerosene Gi2/1 <---cat5---> Gi7/2 lard

In the Guild machine room is a Cisco 4507R 484 watt media converter switch called Lard. This connects to the Cat5/Cat6 above and to single mode fibre, which runs into an ITS managed distribution switch located in the Science library and is imaginatively titled science-dr-01.
lard <----SMF-1000BASE-LX----> science-dr-01 <----SMF-10GBASE-LR?----> UWA-IT <-----SMF-100GBASE-LR?-----> AARNet,world

Machine Room

The machine room contains three switches and a router:

  • Murasoi, a GNU/Linux-based router running iptables/nftables.

  • Walnut, a Ubiquiti EdgeSwitch 16-XG, connecting the router, other switches, and 10Gbps servers

  • Kerosene, a Cisco Catalyst 4506-E running IOS which has a Supervisor 6-E engine, 96 GigE ports and 2 10GE-capable X2 slots.

  • Curviceps, an HP ProCurve 1800-24G which has 24 GigE ports and 2 SFP slots.

These are all labeled and in the rack. There is also a patch panel for the clubroom wall-ports at the bottom of the right-most rack beneath kerosene.

Clubroom

There is Cat5 cabling run from a patch panel at the bottom of the rack to a number of wall ports throughout the room. Where not enough wall ports are available, there are small 5-port unmanaged switches used to attach more devices to the network.

The wireless network is also available in the clubroom.

Layer Two

Internal VLANs

UCC uses seven VLANs internally for various purposes:

  • VLAN 1: Network and server management
  • VLAN 2: Machine room network
  • VLAN 3: Clubroom network
  • VLAN 4: Member VM network
  • VLAN 5: Loft network (used for LANs)
  • VLAN 6: Authenticated wireless network

  • VLAN 7: IoT device network (wired & wireless)

  • VLAN 8: Untrusted wireless network. (deprecated)

  • VLAN 209: UWAES / E-sports network (wired & wireless)

  • VLAN 999: Used as a dummy/blackhole VLAN on some switches. Do not trunk

External VLANs

University IT trunks the following VLAN to us:

  • VLAN 13: Our main uplink, provides us our internet connection and address space

Layer Three

Layer three at UCC is reasonably straightforward these days. A brief summary:

Subnets

There are a number of IP ranges used at UCC for various things:

  • 130.95.13.0/24 is the public address space for our AARNet connection. This range is routed to us via VLAN 13.
    • 130.95.13.0/26 is the machine room address range, internally routed on VLAN 2.
    • 130.95.13.64/26 is the clubroom address range, internally routed on VLAN 3.
    • 130.95.13.128/26 is the member VM address range, internally routed on VLAN 4.
  • 192.168.2.0/24 is the management VLAN IP range. This is not allocated to us by UWA and not routed outside UCC.
  • 192.168.4.0/24 is the private subnet carrying traffic between Murasoi and the 4G modem.
  • 192.168.5.0/24 is the subnet for the "4G backup" Wireguard tunnel between UCC and AWS.

  • 192.168.9.0/24 is the Virtual UCC (VUCC) network (ask [FVP]), routable via vucc0.ucc.asn.au (130.95.13.35)

  • 192.168.11.0/24 is the subnet for the "proxy" Wireguard links between UCC and BinaryLane.

  • 192.168.13.0/24 is allocated to us by UWA and routed on VLAN 13.
    • 192.168.13.0/25 is the uplink network from Murasoi to UWA.
    • 192.168.13.128/25 is the former range for machines colocated in Arts.
      • Currently unused.
  • 192.168.16.0/22 is the authenticated UCC clients range

    • 192.168.16.0/24 is the UCC wifi range

    • 192.168.18.0/24 is the IPsec VPN client range
    • 192.168.19.0/24 is the OpenVPN client range
  • 192.168.20.0/22 is the untrusted / unauthenticated UCC range

    • 192.168.20.0/24 is the new Loft range

    • 192.168.22.0/24 is the IoT range
  • 172.26.42.0/24 is for 'untrusted client machines' and is allocated to us by UWA and routed to us via VLAN 13. There is some history here, but these addresses are not routed outside the Uni. This subnet may be NATted to public IPs for external access.
    • Currently unused.
  • 172.31.0.0/16 is for Docker bridge networks on hosts

Addressing scheme

Most of UCC's subnets use DHCP to assign addresses based on MAC address. Further details can be found at Network/Services#DHCP.

Routing and Firewall

Murasoi, the GNU/Linux router, is a beast of burden. See Network/Firewall for further information on Layer 3 routing and firewalling configuration.

IPv6

http://ipv6.he.net/certification/scoresheet.php?pass_name=accmurphy

UCC has 2405:3C00:5200:100::/58 (which is :100:: to :13f:: inclusive; in other words, 64 networks).

This is an unusually small CIDR block. RFC-6177 recommends that small end sites - such as a home user with devices in the "dozens or less" - should be allocated a /56 block (256 networks).

This is advertised by radvd on Murasoi which most machines autoconfigure from, however some machines have statically assigned addresses. There is an IPv6 firewall that matches our IPv4 firewall very closely.

Many machine room systems have IPv6 address, which are statically assigned. There is no reverse DNS delegation at this stage, so ...ip6.arpa. reverse DNS is UCC-only.

Monnik's DNS record doesn't have an AAAA record, because we are scared of this breaking Things(tm).

IPv6 is routed to 2405:3C00:10:4::1 from Murasoi.

Subnets

  • 2405:3C00:10:4::2/64 uplink (equivalent to 192.168.13.2, UWA VLAN 13)
  • 2405:3C00:5200:100::/64 machine room (VLAN 2)
  • 2405:3C00:5200:101::/64 clubroom (VLAN 3)
  • 2405:3C00:5200:102::/64 member VMs (VLAN 4)
  • 2405:3C00:5200:103::/64 loft (VLAN 5)
  • 2405:3C00:5200:104::/64 wireless (VLAN 6)
  • 2405:3C00:5200:105::/64 IoT (VLAN 7)

  • 2405:3C00:5200:106::/64 public wireless (VLAN 8)

  • 2405:3C00:5200:120::/120 IPsec VPN
  • 2405:3c00:5200:121::/64 OpenVPN
  • 2405:3c00:5200:9100::/64 VUCC "Virtual UCC" IP range (note: not technically owned by UCC and hence not routable from the Internet)

Multicast

UWA runs multicast in sparse PIM mode, and Murasoi runs pimd as noted here. Make sure pimd is only listening once per interface, otherwise things won't work quite right.

Higher Layers

Higher layer configuration at UCC has changed significantly since TheCloudflarening. See that new page for more details.

Monitoring

There are various monitoring packages installed, links to which can be found on MissionControl.

CategorySystemAdministration

uccwiki: Network (last edited 2024-10-14 20:00:05 by 2405:3c00:5200:104:8a3d:73e0:f141:efb9)

MoinMoin Powered | Python Powered
Written to be specifications compliant.

UCC

Wiki content is publicly editable, and as such does not necessarily reflect
the views of the University Computer Club, the Guild or the University of
Western Australia.

Contact the webmasters.