Re: Extracting in real time outgoing/incoming file content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The aoetools are user-space programs that are supposed to make it more
convenient to use ATA over Ethernet.  They include aoeping and
aoe-discover.

The aoe driver is a kernel module that makes remote AoE targets
appear as local block devices like /dev/etherd/e0.1.

The terms "client" and "server" are not used much---data is written
by the initiator to the storage target, usually that means it's
written by the aoe driver to the SRX or vblade, via Ethernet.

On the initiator host, if the userland process using the block device
/dev/etherd/e0.1 opens e0.1 using the O_DIRECT flag, then the kernel
doesn't do the usual page caching.  The page cache normally would
store up all the writes in memory as dirty pages to be written out
to the storage target in a batch later.  But O_DIRECT tells the kernel
to write directly to the storage target.

On 12/31/2016 06:30 PM, NavAil wrote:
> I understand. Does that mean that once the incoming data reaches the
> AOETools layer, it is send immediately, with no latency, to the server
> and no cache mechanism is applied at that layer (and nothing can be done
> at the aoetools level)?
>
>
> ---
> PS
> I am sorry about the unintentional duplicates. I've tried to post via
> Yahoo 10 or more hours ago and it didn't work then, that is why I've
> tried later via another email service. But it seems Yahoo is actually
> having some server problems and sends the emails hours later causing
> this mess.
>
> 31. Dec 2016 21:48 by :
>
>     For item two, you're probably best off bypassing the page cache by
>     opening the device with O_DIRECT.
>
>     On Dec 31, 2016 7:22 AM, some_777@xxxxxxxxxxxx
>     <mailto:some_777@xxxxxxxxxxxx> wrote:
>
>         Hello All,
>
>         I'd appreciate some help on the following  case:
>
>         1. A big file (2-3GB or more) is saved/written at the aoe-client
>         side (to the e0.1 device)
>
>         2. The data written needs to be continuously send over the
>         network to the aoe-server immediately, with no latency
>
>         3. At the server side I need to extract in real time the
>         available incoming raw content fragments (ordered correctly)
>         immediately after they are received and not wait for the whole
>         file to be received, reconstructed and made available via the fs
>
>         What would be the easiest way to do that?
>         I assume some coding would be necessary. What could be done and
>         which are the right driver source files/functions to modify in
>         order to achieve all that?
>
>         Any help would be much appreciated!
>         Thanks
>
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Aoetools-discuss mailing list
Aoetools-discuss@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/aoetools-discuss



[Index of Archives]     [Linux ARM Kernel]     [Linux SCSI]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux