About
UCC has an EMC Clariion CX500 SAN which we was donated to the club by ITS in May 2010. It started life in UCC as a full-rack SAN which was unceremoniously stripped for its parts. We were left with several shelves, or which we decided to use two shelves of 300G disks and the main controller shelves with 176B disks. We also got a couple of fibre channel switches - one is in use and the other is used for spares. [BOB] then went through the pains of breaking into it and setting it up for UCC's use. The setup steps are documented here in the hopes that somebody out there on the internet (or in UCC) will find this information useful one day, as EMC was extraordinarily unhelpful when we were trying to get assistance with setting it up. It is far from complete - a lot of stuff is left to the user to work out once they're into the web interfaces, however this page should get you that far. Sorry but some of the steps are specific to the UCC network - I'm sure you'll figure this out.
The SAN is set up with a switched fabric configuration to the controller shelf, and then the controller shelf is connected to the other disk shelves in an arbitrated loop (usually with a secondary redundant loop). Only the controller shelf (with the Storage Processors) connects to other machines, and any expansion shelves connect to the controller shelf. Think of it as back end and front end fibre (it's labelled as BE and FE on the controller shelf).
FIBRE SWITCH SETUP
The fibre switch that everything is connected to is peanut. Peanut is an EMC/brocade DS-4700 fiber switch. It is a shit of a thing to set up from scratch, mostly due to clariion not releasing any documentation.
- connect to the switch using 115200, 8n1 serial from hyperterminal or putty. Minicom, picocom, decserver, telnet, etc. DO NOT WORK
- login with the password "level-2"
type "PSWD_RESET", this resets the web interface password back to its default
- configure the network settings before leaving (type help to figure this out, it's pretty easy) - put it on the management subnet
open a webpage and go to http://peanut (firefox works fine despite the warning)
- the default username is "Administrator" (case sensitive!) and "password"
- leave the username as Administrator and set the password
Note: as far as we know, ssh is impossible to log into
Note 2: the cool thing about this switch is that you can plug stuff into any port, and it'll route everything for you (once you've set up the devices that should be in a particular zone).
STORAGE PROCESSOR SETUP
The controller shelf is a Clariion CX500, and has two 2.4Ghz xeon servers built into it. This is where shit gets weird. Both servers boot off the same disks, and have a shared bios. They then have separate fibre, ethernet, and even serial ports. Their names are enron and stearns.
- To connect to the controller shelf over serial for initial (bios) setup. For this step you are limited to hyperterminal _only_.
- Settings are:
- 115200bps
- 8n1
- hardware flow control
- put things lile 'terminal emulation' on auto, it should be recognised ok
- the window will look slightly mangled to due to the wrong number of lines and line wrapping, but it's usable for configuring the bios
You then dial in over serial using a ppp connection (same settings as before) to get access to the webpage. Using windows xp this is easy. In Linux see http://tldp.org/HOWTO/PPP-HOWTO/direct.html - no number is required, but you will want the default (serial only) username/password of clariion/clariion! (yes that includes the !). The webpage lets you configure the network settings of the controller, it's available at 192.168.1.1/setup (over serial connection only). Each controller has its own webpage for network configuration, but share a common set of navisphere settings.
- Set up the controller on the management network, making sure to give each controller a DIFFERENT ip. Create a 'root' account and set your storage controller password. You can edit other user accounts once you get to the navisphere web interface.
- Repeat steps 1 to 3 for the second controller
- At this point you should be able to disconnect from the serial and get to the controllers over the network
NAVISPHERE CONFIG
Read the help files on the web interface, they're very educational!
Handy hint: to get rid of error states when they occur and the problem has been remedied, right click on 'UCC_CX500 [CX500]' and select 'Update Now'
Note: default user is admin, but a user 'root' has also been made for UCC use (should probably delete the root user at some point, it's redundant)
STANDBY POWER SUPPLIES
The SAN is equipped with a 1RU Standby Power Supply, with two separate units sharing the same chassis.
SPS's are not UPS's! They will only stay on as long as they need to for the SAN to gracefully shut down. This is usually under a minute.
SPS's are required for the write cache to be enabled, and with 2G write cache it's worth having enabled.
Only one of the SPS's is connected to a storage processor. This is because we only have one cable, and because it tests the SPS every week if it's connected. Since testing the SPS makes the batteries wear, we only connect one, and occasionally switch them so the batteries wear evenly. For some reason, if the cable is connected to the left SPS, it MUST be connected to the left Storage Processor (and similarly for if it's plugged into the right side). Don't try crossing left to right, it gets confused.
