Re: gfapi zero copy write enhancement

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 08/24/2016 03:28 PM, Shyam wrote:
Hi,

I was attempting to review this [1] change, and for a long time I wanted
to understand why we need this and what is the manner in which we need
to achieve the same. As there is a lack of understanding on my part I am
starting with some questions.

1) In writev FOP what is the role/use of the iobref parameter?
- I do not see the posix xlator using this
- The payload is carried in vector, rather than in the iobref
- Across the code, other than protocol/client that (sorf of) serializes
this, I do not see it having any use

So what am I missing?

Looks like this [6] thread has some answers to this, digging through this.


2) Coming to the current change, what prevents us from doing this as [2]?
- in short just pass in the buffer received as a part of iovec

[2] is not necessarily clean, just a hack, that assumes there is a
iovcount of 1 always, and I just tested this with a write call, and not
a writev call. (just stating before we start a code review of the same ;) )

3) From discussions in [3] and [4] I understand that this is to
eliminate copy when working with RDMA. Further, Avati's response to the
thought, discusses why we need to leave the memory management of
read/write buffers to the applications than use/reuse gluster buffers.

So, in the long term if this is for RDMA, what is the change in
justification for the manner in which we are asking applications to use
gluster buffers, than doing it the other way?

4) Why should applications not reuse buffers? and instead ask for
fresh/new buffers for each write call?

5) What is the performance gain noticed with this approach? As the
thread that (re)started this is [5]. In Commvault Simpana,
- What were the perf gains due to this approach?
- How does the application use the write buffers?
  - Meaning, the buffer that is received from Gluster is used to
populate data from the network? I am curious as to how this application
uses these buffers, and where does data get copied into these buffers from.

Eliminating a copy in glfs_write seems trivial (if my hack and answers
to the first question are as per my assumptions), I am wondering what we
are attempting here, or what I am missing.

Thanks,
Shyam

[1] http://review.gluster.org/#/c/14784/
[2] https://paste.fedoraproject.org/413514/14720651/
[3] https://www.mail-archive.com/gluster-devel@xxxxxxxxxxx/msg03347.html
[4]
http://www.gluster.org/pipermail/gluster-devel/2015-February/043966.html
[5] http://www.gluster.org/pipermail/gluster-devel/2016-June/049858.html

[6] http://www.gluster.org/pipermail/gluster-devel/2016-June/049873.html

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel



[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux