Re: high throughput storage server?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



NeilBrown put forth on 3/24/2011 1:33 AM:
> On Thu, 24 Mar 2011 00:52:00 -0500 Stan Hoeppner <stan@xxxxxxxxxxxxxxxxx>
> wrote:
> 
>> If you write a file much smaller than the stripe size, say a 1MB file,
>> to the filesystem atop this wide RAID10, the file will only be striped
>> across 16 of the 192 spindles, with 64KB going to each stripe member, 16
>> filesystem blocks, 128 sectors.  I don't know about mdraid, but with
>> many hardware RAID striping implementations the remaining 176 disks in
>> the stripe will have zeros or nulls written for their portion of the
>> stripe for this file that is a tiny fraction of the stripe size. 
> 
> This doesn't make any sense at all.  No RAID - hardware or otherwise - is
> going to write zeros to most of the stripe like this.  The RAID doesn't even
> know about the concept of a file, so it couldn't.
> The filesystem places files in the virtual device that is the array, and the
> RAID just spreads those blocks out across the various devices.
> 
> There will be no space wastage.

Well that's good to know then.  Apparently I was confusing partial block
writes with partial stripe writes.  Thanks for clarifying this Neil.

> If you have a 1MB file, then there is no way you can ever get useful 192-way
> parallelism across that file.  

That was exactly my point.  Hence my recommendation against very wide
stripe arrays for general purpose fileservers.

> Bit if you have 192 1MB files, then they will
> be spread even across your spindles some how (depending on FS and RAID level)
> and if you have multiple concurrent accessors, they could well get close to
> 192-way parallelism.

The key here being parallelism, to a great extent.  All 192 files would
need to be in the queue simultaneously.  This would have to be a
relatively busy file or DB server.

-- 
Stan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Linux RAID Wiki]     [ATA RAID]     [Linux SCSI Target Infrastructure]     [Linux Block]     [Linux IDE]     [Linux SCSI]     [Linux Hams]     [Device Mapper]     [Device Mapper Cryptographics]     [Kernel]     [Linux Admin]     [Linux Net]     [GFS]     [RPM]     [git]     [Yosemite Forum]


  Powered by Linux