Re: NFS Buffering

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 09/29/2010 02:03 AM, Simon Andrews wrote:
> On 29/09/2010 03:19, JD wrote:
>>
>>> <simon.andrews@xxxxxxxxxxx<mailto:simon.andrews@xxxxxxxxxxx>>   wrote:
>>> The problem is that after scp reports that it's 100% complete the
>>> program will hang for ~20 mins before it will move on to another file.
>>> At this point it can't be killed.
>> Hey! Simon,
>> Listen: buffering is done by the filesystem internals
>> in collaboration with the block io layer. Once the filesystem
>> commits the write to block io layer,  the write call returns to
>> the calling program, and there is not an iota you can do about
>> it!
> I think I get the general process by which the caching happens, and I'm
> not necessarily aiming to get rid of it, just set the cache size to
> something which is appropriate for the speed of the link I'm operating over.
YOU CANNOT!!! IT IS DONE BY THE KERNEL!!! AND IT IS SET IN CONCRETE!!
READ MY SUBSEQUENT REPLY AS WELL.
>
>> In the case of nfs, buffering is done by the nfsiod.
>> Buffering will be done at both the server AND the client.
>> This is especially noticeable when the nfs client writes onto
>> and nfs mounted filesystem.
>> nfsiod is the "helper" kernel thead. There will be as many of these
>> as the admin configures the system for.
> OK, so I'm the admin on the client system.  How to I configure nfsiod?
You can only set the NUMBER of how many to run.
But I have forgotten where it is done - perhaps in
some .conf file in /etc, or a file in /etc/sysconfig/
>
>> Ditto with the main dispatcher,
>> the nfsd process.
>> The nfsiod is what buffers writes on the client side.
> Which means that that is the part which is causing my problems.  The
> main problem is the disparity between the size of the cache on the
> client (somewhere around 2GB) and the speed of transfer onto the NFS
> mounted share (around 2MB/s).  This means that every time the cache
> needs to be flushed there is a ~20min wait during which the client is
> completely blocked (so no chance to kill it or interact with it in any
> way).  Give me a 5MB cache and I'm a happy man
>
>> If you want the scp to function more synchronously, you
>> need to rewrite scp, so that it calls fsync after each write!
>> This will force scp process to wait for the data to be flushed
>> before the write call returns.
> I don't need to be that draconian, just configure a sensible cache size.
>    I just can't see where I can set that.
>
Just give it up dude!
I told you already - cache amount CAN NOT BE CHANGED!!

> Thanks for any advice
>
> Simon.
>
>
>
>
-- 
users mailing list
users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux