We serve out quite a lot of images everyday via our IFrame (online "adult" dating company), and over the past two days we've been running our own in-house file storage system (a replacement for the horribly documented, and terribly buggy MogileFS), and it looks like we're going to end up sticking with NFS. We ran 100 terminal instances doing writes in one test, and reads in another. - 2k writes, one at a time in each instance, averaged from 0.5 to 5 seconds. - 2k reads, again one at a time, averaged from 0.10 to 2 seconds. I don't have a _complete_ log of my tests, but here's a very small portion of the last read test I conducted: /data/hashtest/0/26_IMG_3307_thumb.jpg 1.38440608978 /data/hashtest/0/29_IMG_3399_full.jpg 1.51509380341 /data/hashtest/0/28_IMG_3393_thumb.jpg 0.752840042114 /data/hashtest/0/28_IMG_3381_thumb.jpg 1.84753394127 /data/hashtest/0/29_IMG_3444_thumb.jpg 2.16324305534 /data/hashtest/0/27_IMG_3283_full.jpg 2.13876509666 /data/hashtest/0/27_IMG_3337_full.jpg 1.30814599991 /data/hashtest/0/29_IMG_3423_thumb.jpg 1.22093510628 /data/hashtest/0/26_IMG_3372_thumb.jpg 1.3562169075 /data/hashtest/0/26_IMG_3358_full.jpg 1.38488101959 /data/hashtest/0/26_IMG_3418_full.jpg 1.2317378521 /data/hashtest/0/25_IMG_3356_full.jpg 2.0598461628 /data/hashtest/0/26_IMG_3437_full.jpg 1.93640899658 /data/hashtest/0/29_IMG_3322_full.jpg 1.55558204651 /data/hashtest/0/31_IMG_3329_full.jpg 2.05298900604 /data/hashtest/0/26_IMG_3376_thumb.jpg 1.09549021721 /data/hashtest/0/27_IMG_3443_thumb.jpg 1.72550582886 /data/hashtest/0/25_IMG_3320_full.jpg 1.33485102654 /data/hashtest/0/25_IMG_3362_thumb.jpg 1.07689189911 /data/hashtest/0/26_IMG_3319_full.jpg 1.78225803375 /data/hashtest/0/24_IMG_3422_thumb.jpg 1.41833090782 /data/hashtest/0/29_IMG_3418_thumb.jpg 0.853131055832 /data/hashtest/0/26_IMG_3481_full.jpg 1.51521205902 /data/hashtest/0/28_IMG_3419_thumb.jpg 1.88594007492 /data/hashtest/0/27_IMG_3333_thumb.jpg 1.11501097679 /data/hashtest/0/28_IMG_3464_full.jpg 1.73386716843 /data/hashtest/0/26_IMG_3329_thumb.jpg 0.14195895195 /data/hashtest/0/26_IMG_3349_full.jpg 0.453220129013 /data/hashtest/0/24_IMG_3399_thumb.jpg 0.319472074509 /data/hashtest/0/28_IMG_3357_thumb.jpg 0.184777975082 /data/hashtest/0/29_IMG_3318_thumb.jpg 0.200763940811 /data/hashtest/0/28_IMG_3349_thumb.jpg 0.217283010483 /data/hashtest/0/29_IMG_3465_full.jpg 0.357889890671 /data/hashtest/0/28_IMG_3434_full.jpg 0.686009883881 /data/hashtest/0/29_IMG_3415_full.jpg 0.468312978745 /data/hashtest/0/29_IMG_3325_thumb.jpg 0.15156006813 /data/hashtest/0/27_IMG_3345_thumb.jpg 0.261151075363 /data/hashtest/0/27_IMG_3481_thumb.jpg 0.219217061996 /data/hashtest/0/29_IMG_3338_full.jpg 0.488296985626 /data/hashtest/0/26_IMG_3360_full.jpg 0.263535022736 /data/hashtest/0/24_IMG_3371_thumb.jpg 0.23340010643 /data/hashtest/0/27_IMG_3396_thumb.jpg 0.272875070572 /data/hashtest/0/26_IMG_3402_thumb.jpg 0.241803884506 /data/hashtest/0/24_IMG_3337_thumb.jpg 0.191148042679 /data/hashtest/0/24_IMG_3335_thumb.jpg 0.410676956177 /data/hashtest/0/26_IMG_3387_full.jpg 0.310095071793 /data/hashtest/0/27_IMG_3405_thumb.jpg 0.165282964706 /data/hashtest/0/26_IMG_3429_full.jpg 0.305140018463 /data/hashtest/0/29_IMG_3349_thumb.jpg 0.158540010452 /data/hashtest/0/22_IMG_3337_thumb.jpg 0.422141075134 /data/hashtest/0/29_IMG_3358_thumb.jpg 0.363726854324 /data/hashtest/0/28_IMG_3396_full.jpg 0.286473035812 /data/hashtest/0/28_IMG_3406_full.jpg 0.244587898254 /data/hashtest/0/27_IMG_3454_thumb.jpg 0.144367218018 /data/hashtest/0/24_IMG_3413_full.jpg 0.253911018372 GlusterFS shows a lot of promise, but right now I its current version I don't feel that it's fast enough for us to serve out images.
