based on Solaris http://openindiana.org
FreeBSD https://www.freebsd.org
IF you just want something totally strange and different
there is inferno os ( a derivative of Plan 9 os) more info http://www.vitanuova.com/inferno/
However amazing Arch Linux is, it's certainly the wrong choice for a NAS.
The first thing to consider is how you're doing RAID. Considering the fact that I don't see a large raid card in there, you're either using the Mobo's raid or software raid. If you're using the Mobo raid, often called "fakeraid", you should immediately change that. That is a horrible idea. It has no benefits over software raid, and multiple downsides.
If you're doing software raid, it's basically unquestionable that ZFS is the best option. ZFS-5 and ZFS-6 are absolutely amazing. btrfs is working to reach feature parity there, but it's so unstable I wouldn't trust any data to it.
Since you're running arch linux, you can't have ZFS integrated into the kernel for licensing reasons. Therefore, you're probably stuck using mdadm... Which is definitely inferior. You can use ZFS in userspace in linux, but it's neither as stable/trusted nor as good an idea. I certainly don't want to use userspace drivers for something as core as the filesystem.
The only sane way to get ZFS is to run a BSD or Solaris fork. OpenIndiana and other illumos based Solaris-forks are great and that's what I'd recommend.
I don't mean to come off as an asshole or arch-hater, but I put myself in the exact situation in the past of using linux+mdadm instead of zfs, and I've regretted it since. It might be worth switching if you still have very little data on your NAS.
The hardware looks great though!
http://openindiana.org/ is the latest in the Solaris line, featuring zfs. zfs has dedup built in at the filesystem layer, as well as a whole slew of other features: mirroring, RAIDZ, easy filesystem expansion, and maybe more you don't care about. OpenIndiana bundles samba and kernel level CIFS sharing, and the usual suspects (an ftpd, sshd, iscsi target stack, yadda yadda.)
ZFS is available in Linux as zfs-fuse. It will be slower than a native filesystem. It's also available in FreeNAS if you speak BSD.
Depends on what you're wanting to do. Point to point FC isn't really going to teach you much. As for an OS. Windows Server, OpenIndiana. http://openindiana.org/
8GB FC is still somewhat expensive, at that price you might as well get a 10GBe cards on ebay.
If you really want to learn FC you need to buy a FC Switch. Brocade DS-5100B, Brocade 200E/300E are great cheap options. You can find the same switches rebranded by EMC,HP,Dell
A limited liability way to see what you think about BYO could be to build a small one as a test. OpenFiler, NAS4Free, and OpenIndiana are probably worth a look. I've been looking on these for my home lab for a while, but haven't gotten to test any of the OSes yet.
Ever look at OpenSolaris* while it was still around? A lot of the utilities (including their packaging system) are written in python - same with Solaris 11 Express.
Given Solaris runs on sparc (almost by definition), they're gonna want as much python performance as they can get, so I'd be very surprised if there wasn't some interest getting PyPy working for sparc.
*It's been forked to OpenIndiana if you still want to play with it
Solaris under the Solaris name is dying. Illumos, which forked from OpenSolaris, is much more interesting. You've got OmniOS for bare metal servers, SmartOS for hypervisors, and OpenIndiana for desktops.
> do you by any chance happen to have good resources/Information regarding the downsides of hosting your fileservers/storage via Solaris instead of Linux or FreeBSD?
Not really, i have never used Solaris. All i know it's than Solaris generally use the ZFS file system witch is supported by both Linux and FreeBSD. ZFS have a very good reputation, and major NAS distributions use it by default (FreeNAS, NAS4Free). OpenMediaVault can use it too.
So no "downsides" for the file system itself (if ZFS). If your problem is the fact than Solaris is proprietary, then have a look to Openindiana & Illumos the main OpenSolaris forks. You can also use Linux or FreeBSD with the ZFS file system.
I'm really not sure how it is nowadays, but it has zfs support. Last I checked Linux only had minor support for zfs.
It used to be that opensolaris had the best zfs support, freebsd had 2nd best, and linux had a fuse module.
In terms of usability, I've tried opensolaris in the past and found it painfully slow and bloated compared with linux.
edit: found a good overview of what it offers http://openindiana.org/support/documentation/