Molmol is the File Server. See also: MoneyMoneyMoney/NewStorage, where it was motivated and specc'd out.
Hardware
The "Sick of your moaning" design is *almost* what we have. Except the PSU, case, SAS expander card, and (hopefully soon) an SSD bay. It is basically the same. People who care to correct this.
System Drives
There are 2 SSDs, partitioned with a GUID Partition Table (GPT).
Partition 1 on both stores the boot loader (FreeBSD's gptboot).
Partition 2 on both contains a FreeBSD GEOM MIRROR (with a GPT type of freebsd-ufs) containing the root file system.
- Partition 3 on both forms a mirror for the ZFS SLOG (journal).
- Partition 4 on both forms a spanned (i.e. not mirrored) ZFS L2ARC (disk cache).
- Partition 3 on both are two swap partitions, both of which are enabled as separate swap devices (i.e. spanned, not mirrored).
root@molmol ~# gpart show
# Start Size Part# Type
=> 34 500118125 ada0 GPT (238G)
34 1000 1 freebsd-boot (500K)
1034 1014 - free - (507K)
2048 79691649 2 freebsd-ufs (38G)
79693697 4192385 5 freebsd-swap (2.0G)
83886082 4194304 3 freebsd-zfs (2.0G)
88080386 412037773 4 freebsd-zfs (196G)
=> 34 500118125 ada1 GPT (238G)
34 1000 1 freebsd-boot (500K)
1034 1014 - free - (507K)
2048 79691649 2 freebsd-ufs [bootme] (38G)
79693697 4192385 5 freebsd-swap (2.0G)
83886082 4194304 3 freebsd-zfs (2.0G)
88080386 412037773 4 freebsd-zfs (196G)The GEOM MIRROR RAID is monitored by setting daily_status_gmirror_enable="YES" in /etc/periodic.conf.
ZFS
Assembled the RAID along the lines of https://pthree.org/2012/04/17/install-zfs-on-debian-gnulinux/
root@molmol:~# zpool status
pool: space
state: ONLINE
scan: scrub repaired 0 in 2h9m with 0 errors on Tue Oct 21 21:30:01 2014
config:
NAME STATE READ WRITE CKSUM
space ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
mirror-3 ONLINE 0 0 0
da6 ONLINE 0 0 0
da7 ONLINE 0 0 0
logs
mirror-4 ONLINE 0 0 0
gpt/molmol-slog ONLINE 0 0 0
gpt/molmol-slog0 ONLINE 0 0 0
cache
gpt/molmol-l2arc ONLINE 0 0 0
gpt/molmol-l2arc0 ONLINE 0 0 0root@molmol:~# zfs list NAME USED AVAIL REFER MOUNTPOINT space 1.73T 1.83T 200K /space space/away 909G 1.83T 909G /space/away space/mp3s 281G 1.83T 281G /space/mp3s space/scratch 192K 1.83T 192K /space/scratch space/services 585G 1.83T 585G /space/services space/vmstore 144K 1.83T 144K /space/vmstore
Compression is on: zfs set compression=lzjb space - you can see how well it's working with zfs get compressratio
Other notes
In order to prevent a hang on shutdown, the following (sourced from PCBSD)has been added to /etc/sysctl.conf:
# Don't hang on shutdown when using USB disks hw.usb.no_shutdown_wait=1
HOW DO I FREEBSD
Don't panic. The documentation is super good - start with The FreeBSD Handbook.
There is a clear separation between "base system" and "additional software" in FreeBSD - more along the lines of how Windows does it. Most "additional software" gets installed and configured in /usr/local, so if in doubt, look there (e.g. /usr/local/etc).
Install new software with pkg(8) (e.g. pkg install zsh).
Configuration of base system services is done with /etc/rc.conf, and additional software services in /usr/local/etc/rc.conf.
Use zpool and zfs to manipulate ZFS - see the ZFS on Linux tutorial for a good introduction to the concepts.