I have a new baby

Well, my home network does, at least. Marrow is my 1 week old media storage server, and has almost a terabyte of storage space - which is exported via nfs to the other machines in my house.

The machine itself is an old Dell Dimension XPS. The “XPS” is Dell-speak for “not quite as shit as a standard Dimension”. I’ve always been a fan of Dell Optiplex and Precision desktops - but the Dimension and XPS series make the Slashdot crowd’s sneery attitude towards Dell desktops seem justified. ah well.

Anyway, when bought, marrow had a 700 MHz Pentium 3 cpu, 256 MB RAM, 20 GB of hard disks, and a quad speed cd-rom drive. Now, it has er, a 700 MHz cpu, 756 MB RAM, 5x 250MB disks, 2x five disc cd changers, and a dvd-rw drive. Anyone would think that it’d been pilfering from a cyclist’s medicine cabinet.

Each disk has a two partitions on it, of 5 GB and 245 GB:

+------------++------------++- - - - - - +   +------------------------+
|            ||            |             |   |                        |
| /dev/hda1  || /dev/hdb1  || /dev/hdc1      | /dev/hdd1    /dev/hde1 |
| raid 1     || raid 1     |  raid 1     |   | raid 0                 |
| / (mirror) || / (mirror) || / (spare)      | swap                   |
|            ||            |             |   |                        |
+------------++------------++ - - - - - -+   +------------------------+

+---------------------------------------------------------------------+
|                                                                     |
| /dev/hda2     /dev/hdb2      /dev/hdc2       /dev/hdd2    /dev/hde2 |
| raid 5                                                              |
| mounted on /srv                                                     |
|                                                                     |
+---------------------------------------------------------------------+

So, we have a redundant, mirrored 5 GB root partition, with a hot spare. 10 Gb of swap is available on striped set - this is greeat for a toy setup, but a silly idea for a real machine, as failure of either disk will cause the machine to die spectacularly if/when it tries to use more than half of the available swap space.

I’ve been fiddling around to see the best parameters to use with nfs. After lots of geeky testing, I’ve come up with the following mount line:

marrow:/srv/music /home/rory/music nfs rw,noatime,rsize=32768,wsize=32768,hard,intr

noatime: Don’t update the file’s last access timestamp - I’ve never used the atime for anything, so I’m happy not to have to to the remote server every time I touch a file.

rsize, wsize: These values surprised me. The default is 2048, and the standard recommendation is to increase it to 8192 if your server supports large values. To be honest, I’d expected that 64k - the server’s raid 5 stripe size - would provide the best performance; but, in my tests, 32k was significantly faster than any other value (with 16k being marginally faster than either 8k or 64k). I have a feeling that this is because the mode of the file sizes in my trial set (my homedir, essentially) was 31.5k. In other words, the optimal size of the rsize and wsize values are dependant on the size of the files being used.

hard: This allows file operations to be retried even if they initially time out - which means that the nfs server can go away for a reboot without causing any client-side problems

intr: hard mounting has a drawback, however - if the server crashes, the client will keep on trying to access it. You won’t be able to close any program that has a lock on a remote file, or unmount any remote partitions. intr fixes this by letting the client interrupt any such operation. hurrah!

Finally, a note about using nfs on a desktop system. Desktop enivronments, like GNOME, need to know whats happening to files in a users homedir, so that they can show those changes in the file manager, etc.

The obvious way to do this is to periodically poll each file for changes. This is, of course, slow, system intensive, and icky. A much better way is for the kernel to notify the desktop each time that some other subsystem changes a file - something it does using the older dnotify, or shiny new inotify interface.

In the olden days, we used “fam” to listen for dnotify events. You could run it on a remote nfs server, and get it to broadcast update notifications to nfs client machines, which was all very handy. However, fam has now been replaced with “gamin”, which is more lightweight (and so less system-intensive), and can listen for inotify events rather than dnotify (again, less system-intensive).

Unfortunately, gamin can’t do the funky listen-to-a-remote-fam-server thing. This is supposedly a feature-not-a-bug. “security, donchaknow”. BUT, JESUS FUCK, NFS IS WILDLY INSECURE /ANYWAY/. cretins.

So, gamin falls back to polling all your nfs mounts every 10 seconds. Braindead, yes? So, ease the pain by making it every 90s instead, by editing your /etc/gamin/gaminrc to add the following line:

fsset nfs poll 90

With that added, marrow is able to saturate the network to both humble (full duplex eth100) and sanguine (802.11g) - which is exactly what I wanted to achieve. hurrah!