Re: NFS for millions of files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Sep 03, 2009 at 02:37:51PM -0400, Ric Wheeler wrote:
> On 09/03/2009 02:15 PM, J. Bruce Fields wrote:
>> On Wed, Sep 02, 2009 at 02:37:31PM -0400, Peter Staubach wrote:
>>    
>>> Please keep in mind that the NFS stable storage requirements are
>>> probably causing a significant slowdown in activities such as this.
>>>      
>> My first thought too, but:
>>
>>    
>>> Jason Legate wrote:
>>>      
>>>> When I run our creation benchmark locally  I can get around 3000
>>>> files/ second in the configuration we're using now, but only around
>>>> 300/second over NFS.  It's mounted as this:
>>>>        
>> ...
>>    
>>>> When I mount the same FS over localhost instead of across the lan,
>>>> it performs about full speed (the 3000/sec).
>>>>        
>> The localhost NFS mount would be incurring the same sync latency, so all
>> his latency must be due to network.  (And with those numbers I guess
>> he's either got lots of disk spindles, or an ssd, or (uh-oh) has the
>> async option set?)
>>
>> --b.
>>    
>
> For small files without doing an fsync per file, getting 3000 files/sec  
> is not that much. Ext3 can do it with a local s-ata disk. I suspect that  
> Jason would run much slower if he ran with local fsync()'s enabled  
> (similar to what NFS servers have to do).

Yeah, which is why I was suspecting they set "async" on the export.  In
which case I hope they know what they're doing....

--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux