Re: another proposed protocol optimization...

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



You beat me by a few days on this. I was looking at something similar 
some time ago, but I was thinking a bit in another direction.
I'm a bit verbose today, if you missed that from the previous mails, but 
please bare with me.
tl;dr
- 'negotiation' should be 'discovery' by initiator queries only
- check with Coraid for their own feature detection method(if applicable)
- I propose a feature string based discovery method
- your coalesced read suggestion sounds confusing, please document and 
split to vendor command


------------------On feature discovery------------
Since Coraid basically controls the protocol and they make actual 
commercial products against it, it would be only natural to synchronize 
with them on this before making any changes to it.
I don't have access to traffic dumps or actual hardware, so I'd like to 
first ask Ed Cashin if Coraid has implemented and/or uses any vendor 
specific extensions that he can talk about.
Regarding 'feature negotiation', I believe that this cannot be used as 
the proper term here. Since AoE is connectionless, it would incur that 
the server should not care or have any state about a potential client. 
As such, I believe the onus should be on the initiator to determine what 
the target supports and adapt to that. I believe 'feature detection' is 
probably a better term.
To this end, since there is no requirement that both initiator and 
target be written by the same vendor, I believe that last point of AoE 
r11 point 2.6. Commands (Command Codes 0xf0-0xff are reserved for vendor 
specific use) should be considered invalid without a proper, sanctioned, 
mechanism in place to prevent collision of vendor specific uses that 
could have dangerous side-effects.

Looking at command 1 (Query Config Information) there is nothing that 
would allow for safe detection of features.
Providing vendor identification inside a config string could be lost in 
normal usage. Obtaining the vendor/feature list from the ATA IDENTIFY 
DEVICE command (ECh) might not be reliable for devices that actually do 
ATA pass-trough to the respective drives (I've read about some 
enclosures that expose the drive over AoE and they might want to do that).

I was looking to propose the following change
1. define a feature as
- a string of lower-case ascii letters representing the feature
- vendors features must be prefixed with a consistent (for that vendor) 
upper-case ascii letters identifier
- features must have at most 15 characters
2. define Feature String as a comma separated list of at most 32 
features (with a total maximum length of 512 bytes)
4. define Feature Response as a string of at most 31 bytes formed by 
dash joining of
- a feature
- a hex number representing the command code the target uses for above 
feature
- an optional argument(maximum of 12 alphanumeric chars)
4. define a Feature Response String as a comma separated list of at most 
32 feature responses (with a total maximum length 1024 bytes)
5. allocate Command 1. CCmd 0x5(or 0xF) as 'test feature string'
CCmd 5: test feature string
  Respond with a Feature Response String containing features responses 
for features valid on this target.
  Response could be empty(and would return an error if CCmd 0x5 is 
unsupported)
6. An initiator should use this as follows.
- iterate own supported features
- initiate a query for CCmd 5 with argument set to feature string
- parse response and configure itself to use discovered features on the 
command codes provided in the feature response string
- use supported features

-------------On coalesced read requests------------
I'm quite interested in how you manage to do this in WinAoE (from a 
previous mail, I understand you are working with that as initiator) 
without changing quite a bit of the logic, given that the Tag is 
generated in the network scheduling code, not in the disk code.
I would ask that you move this code to a vendor reserved command code, 
unless it's included in the official protocol.
In the same style you propose here, it would be interesting to 
concatenate several write responses into fewer ones, since they are all 
fired towards the target at once by the initiator.
My concerns with this kind of code have less to do with cpu usage and 
more to do with latency. Sending one packet instead of multiple would 
reduce the waiting on both ends. Receiving responses for writes would 
also benefit due to the spinlock on the tag list on reply.
As an alternative that may be slightly easier to implement and manage, I 
was thinking of sending only one read request for several sectors(set 
sectors to the original number in the Disk IO request), while making it 
clear to the target that they should be sent back with the same Tag but 
in separate frames. Responses should fill the IRP buffer until it is 
complete and only one IRP would need to be completed. This could allow 
for less memory usage for the initiator, as less memory would be used to 
track requests. One side effect would be that the timeout routine 
currently used by WinAoE to reschedule missed packets would not be aware 
of what parts of the buffer are missing. This would either mean 
resetting the received counter and rescheduling all requests (or storing 
a bitmap of them) or else re-request the entire buffer. The second 
option would cost quite a bit of time if the requested area is large.

--The end!--
I'd say that this was my 2 cents, but it looks more like 2 dollars..
Thanks for reading!


On 10/06/2014 12:18 AM, Killer{R} wrote:
> Hello aoetools-discuss,
>
>    ..coalesced read requests. Its quite an expensive to send 60-bytes
>    packet per every MTU-sized read. I did experimental read requests
>    coalesting in my forked project and it noticable decreased
>    packets-per-second rate and also reduced target's CPU usage.
>    I did this by appending bunch of such structures prepended by uchar
>    indicated their count just after normal Read command packet:
>    struct AtaCoalescedRead
>    {
>          uchar   tag[4];
>          uchar   lba[6];
>          uchar   resvd[5];
>          uchar   sectors;
>    };
>    + added feature negotiation command so initiator can negitiate with
>    target using of that improvement without losing backward compatibility
>
>


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Aoetools-discuss mailing list
Aoetools-discuss@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/aoetools-discuss




[Index of Archives]     [Linux ARM Kernel]     [Linux SCSI]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux