Hi,
I'm sorry to say it was not clear to me that you went ahead with a fork
from your previous mail. (but given the time I took to answer it, I
should have assumed that)
I'm not exactly working in a resource constrained environment and I
already have most of the data cached in RAM. As I attempted to say
earlier, my main issue with these changes being in a fork is that they
stand a much smaller chance of being included in a distribution for
something that might need it (say Raspbian, for example. vblade is in
the Debian repository) and is also less likely to get contributions and
support from people that might care about the changes, without at least
a reference from the vblade README file.
However, I would like to encourage you to split at least the read-ahead
code and make a pull-request.
Regarding the mentioned WinAoE deadlock, there are no crashes or stack
dumps in the described situation. The issue I was talking about
manifests itself when some filters attach to the stack. Windows appears
to like to unload filters from the stack when inserting some other types
of filters, including some firewall products. This might not be
noticeable when using AoE for secondary storage, but when the system
drive is affected, Windows removes the AoE filter before attaching the
new driver to the stack, then attempts to save the changes to disk
before reattaching AoE. It is somewhere between these changes that the
system locks until the disk flushes the changes, which is, of course,
dependant on the filter being attached to the network card. This causes
a deadlock. On Windows XP, the system would die after a few seconds. On
Windows Vista and above, including with the NDIS6 network rewrite, it
will hang there for quite a bit of time, acting like the disk is
faulting. Some caching might help with this, but it wasn't high enough
on the priority list for me to test if Windows would then ask for a read
from the disk, that would prove caching was largely useless. (our
specific usage pattern would actually suffer from constant caching in
WinAoE)
I haven't yet looks at the linked projects, but I believe you should ask
the previous developer of WinAoE, if you still can, if they're ok with
you using the same name for the fork.
Cheers!
On 11/06/2014 1:52 AM, Killer{R} wrote:
> Hello Catalin,
>
> Tuesday, June 10, 2014, 4:19:21 PM, you wrote:
>
>
> At first as I said I already forked it:
> vblade's child: https://sourceforge.net/projects/aoede/
> WinAoE: https://sourceforge.net/projects/winaoe/
> Brief changes description (including extension negotiation you asked
> in another message) can be found on Wiki links of both projects.
>
> A bit more about AIO and its performance impact. Documentation tells
> that AIO actually works only with O_DIRECT enabled. This means that if
> you use AIO for a file descriptor opened without O_DIRECT - operations
> will be performed synchrnonously and there will be no difference at
> all with 'usual' read/write. But using O_DIRECT means that file IO
> will not be cached by Linux kernel that normally makes them much slower.
> So AIO is reasonable only if you implemented some own caching
> functionality that is somewhy better than Linux'es (or you dont want
> caching at all). That approach was good in my usecase - on the system
> with limited CPU and RAM: at first buffering in userspace seems improved
> write performance even without using AIO/direct mode. Not sure why,
> but difference was about twice. Futher using of O_DIRECT and AIO
> improved write performance even more (obviously due to avoided memory
> copying between my and OS' caches). Also I implemented read-ahead
> buffering to keep read performance same good as without O_DIRECT/AIO.
> Good thing in all this is that such way of using file doesn't bother
> system-wide cache, and my caching uses very small amount of RAM - 3
> megs by default config (may be in more 'thicker' system its reasonable
> to add buffers - its possible by editing tuneup.h). And even that 3
> megs of RAM are freed after 10 seconds of inactivity.
> About FreeBSD - FreeBSD doesnt support Linux AIO, but there is another
> AIO, Posix AIO, that a bit different. Linux however supports posix AIO
> but via 'simulation' - GLib runs userspace threads to perform everything
> using usual read/write, and performance is discouraging. Also FreeBSD has
> different implmeentation of socket mapped ring buffer, so in my fork
> I decided to do not care about FreeBSD, lucky if it still compiles for it :)
>
> About WinAoE: I was able to contact its author, and he told me that he
> dropped project support due to SSD technology that by his opinion
> eliminated need for AoE: cuz now systems not should be diskless to work
> noiseless. Anyway, my changes almost doesn't affect networking part,
> only scheduler was changed and added support for AoEde's features.
> About your deadlock - I could fixed it.. occasionally :) Its hard to
> say definately without looking at dump/stack/
>
> CS> Sorry for the delay. I missed this thread. I'd like to chip-in here.
>
> CS> @Ed Cashin
> CS> I slightly disagree with the need to fork this. I believe it should go
> CS> into the contrib folder, should it be kept up to date.
>
> CS> The previous AIO patch is documented to have offered negligible
> CS> improvements on Linux at that time.
> CS> I contacted the patch writer some time ago and enquired about the
> CS> results. As I recall, it also offered similarly disappointing results on
> CS> FreeBSD (but then again the old code did not use jumbo-frames)
> CS> Dropping the new code in the contrib folder as a patch sounds better to
> CS> me (and maybe if it is sufficiently stable, maintainers might wish to
> CS> include it in distributions)
>
------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Aoetools-discuss mailing list
Aoetools-discuss@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
