|
Hi all, I bid adieu to you all with the hope of crossing path again, and the time has come rather quickly. It feels great to work on GlusterFS again. Currently we are trying to write data backed up by Commvault Simpana to glusterfs volume (Disperse volume). To improve the performance, I have implemented the proposal put forward my Rafi K C [1]. I have some questions regarding libgfapi
and iobuf pool. To reduce an extra level of copy in glfs read and write, I have implemented few APIs to request a buffer (similar to the one represented in [1]) from iobuf pool which can be used by the application to write data to. With this implementation,
when I try to reuse the buffer for consecutive writes, I could see a hang in syncop_flush of glfs_close (BT of the hang can be found in [2]). I wanted to know if reusing the buffer is recommended. If not, do we need to request buffer for each writes? Setup : Distributed-Disperse ( 4 * (2+1)). Bricks scattered over 3 nodes. [1]
http://www.gluster.org/pipermail/gluster-devel/2015-February/043966.html [2] Attached file - bt.txt Thanks & Regards, Sachin Pandit. ***************************Legal Disclaimer*************************** "This communication may contain confidential and privileged material for the sole use of the intended recipient. Any unauthorized review, use or distribution by others is strictly prohibited. If you have received the message by mistake, please advise the sender by reply email and delete the message. Thank you." ********************************************************************** |
(gdb) thread apply all bt
Thread 9 (Thread 0x7fa60eada740 (LWP 16211)):
#0 0x00007fa60cc546d5 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007fa60e2100b9 in syncop_flush (subvol=0x7fa5f402e650, fd=0x19da74c, xdata_in=0x0, xdata_out=0x0)
at syncop.c:2359
#2 0x00007fa60e4ab3b3 in pub_glfs_close (glfd=0x19da670) at glfs-fops.c:270
#3 0x0000000000400b48 in main (argc=<optimized out>, argv=<optimized out>) at libgfapi_zero.c:130
Thread 8 (Thread 0x7fa5f932d700 (LWP 16219)):
#0 0x00007fa60d87c7f3 in epoll_wait () from /lib64/libc.so.6
#1 0x00007fa60e229f53 in event_dispatch_epoll_worker (data=0x7fa5f405f670) at event-epoll.c:664
#2 0x00007fa60cc50dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007fa60d87c21d in clone () from /lib64/libc.so.6
Thread 7 (Thread 0x7fa601a30700 (LWP 16218)):
#0 0x00007fa60cc55225 in pthread_spin_lock () from /lib64/libpthread.so.0
#1 0x00007fa60e1f373e in iobref_unref (iobref=0x19dc7e0) at iobuf.c:907
#2 0x00007fa60e246fb2 in args_wipe (args=0x19e70ec) at default-args.c:1593
#3 0x00007fa60e1ea534 in call_stub_wipe_args (stub=0x19e709c) at call-stub.c:2466
#4 0x00007fa60e1ea5de in call_stub_destroy (stub=0x19e709c) at call-stub.c:2482
#5 0x00007fa5fad822bb in __wb_request_unref (req=0x19e8860) at write-behind.c:369
#6 0x00007fa5fad83279 in __wb_fulfill_request (req=0x19e8860) at write-behind.c:677
#7 0x00007fa5fad8370d in wb_head_done (head=0x19e8860) at write-behind.c:831
#8 0x00007fa5fad83c14 in wb_fulfill_cbk (frame=0x19d88cc, cookie=0x19e4eec, this=0x7fa5f4026010, op_ret=131072,
op_errno=0, prebuf=0x7fa5f409cdfc, postbuf=0x7fa5f409ce6c, xdata=0x7fa5f409cbdc) at write-behind.c:1000
#9 0x00007fa5fb001f30 in dht_writev_cbk (frame=0x19e4eec, cookie=0x19e501c, this=0x7fa5f40246e0, op_ret=131072,
op_errno=0, prebuf=0x7fa5f409cdfc, postbuf=0x7fa5f409ce6c, xdata=0x7fa5f409cbdc) at dht-inode-write.c:106
#10 0x00007fa60e2607a1 in default_writev_cbk (frame=0x19e501c, cookie=0x19ee75c, this=0x7fa5f401f9f0,
op_ret=131072, op_errno=0, prebuf=0x7fa5f409cdfc, postbuf=0x7fa5f409ce6c, xdata=0x7fa5f409cbdc)
at defaults.c:1242
#11 0x00007fa5fb27284d in ec_manager_writev (fop=0x19ee75c, state=5) at ec-inode-write.c:1544
#12 0x00007fa5fb24ea4b in __ec_manager (fop=0x19ee75c, error=0) at ec-common.c:2204
#13 0x00007fa5fb24a4d2 in ec_resume (fop=0x19ee75c, error=0) at ec-common.c:289
#14 0x00007fa5fb24a6fa in ec_complete (fop=0x19ee75c) at ec-common.c:362
#15 0x00007fa5fb26c281 in ec_inode_write_cbk (frame=0x19e514c, this=0x7fa5f401f9f0, cookie=0x2, op_ret=65536,
op_errno=0, prestat=0x7fa601a2f900, poststat=0x7fa601a2f890, xdata=0x7fa5f409cbdc) at ec-inode-write.c:64
#16 0x00007fa5fb271f0b in ec_writev_cbk (frame=0x19e514c, cookie=0x2, this=0x7fa5f401f9f0, op_ret=65536,
op_errno=0, prestat=0x7fa601a2f900, poststat=0x7fa601a2f890, xdata=0x7fa5f409cbdc) at ec-inode-write.c:1405
#17 0x00007fa5fb4fab53 in client3_3_writev_cbk (req=0x7fa5f0010bbc, iov=0x7fa5f0010bfc, count=1,
myframe=0x7fa5f000c56c) at client-rpc-fops.c:859
#18 0x00007fa60df8431b in rpc_clnt_handle_reply (clnt=0x7fa5f408d650, pollin=0x7fa5f4001850) at rpc-clnt.c:759
#19 0x00007fa60df847ef in rpc_clnt_notify (trans=0x7fa5f408db30, mydata=0x7fa5f408d680,
event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7fa5f4001850) at rpc-clnt.c:920
#20 0x00007fa60df80b85 in rpc_transport_notify (this=0x7fa5f408db30, event=RPC_TRANSPORT_MSG_RECEIVED,
data=0x7fa5f4001850) at rpc-transport.c:541
#21 0x00007fa60101e02d in socket_event_poll_in (this=0x7fa5f408db30) at socket.c:2349
#22 0x00007fa60101e56a in socket_event_handler (fd=14, idx=5, data=0x7fa5f408db30, poll_in=1, poll_out=0,
poll_err=0) at socket.c:2462
#23 0x00007fa60e229b68 in event_dispatch_epoll_handler (event_pool=0x1992680, event=0x7fa601a2fee0)
at event-epoll.c:571
#24 0x00007fa60e229f85 in event_dispatch_epoll_worker (data=0x7fa5fc000920) at event-epoll.c:674
#25 0x00007fa60cc50dc5 in start_thread () from /lib64/libpthread.so.0
#26 0x00007fa60d87c21d in clone () from /lib64/libc.so.6
Thread 6 (Thread 0x7fa602231700 (LWP 16217)):
#0 0x00007fa60cc51ef7 in pthread_join () from /lib64/libpthread.so.0
#1 0x00007fa60e22a1a7 in event_dispatch_epoll (event_pool=0x1992680) at event-epoll.c:758
#2 0x00007fa60e1edd2c in event_dispatch (event_pool=0x1992680) at event.c:124
#3 0x00007fa60e4a5dfb in glfs_poller (data=0x191c010) at glfs.c:612
#4 0x00007fa60cc50dc5 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa60d87c21d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7fa602c35700 (LWP 16216)):
#0 0x00007fa60d84342d in nanosleep () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit---
#1 0x00007fa60e1cd09b in gf_timer_proc (data=0x199b7a0) at timer.c:176
#2 0x00007fa60cc50dc5 in start_thread () from /lib64/libpthread.so.0
#3 0x00007fa60d87c21d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7fa60995d700 (LWP 16215)):
#0 0x00007fa60cc54a82 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007fa60e204046 in syncenv_task (proc=0x1995c30) at syncop.c:603
#2 0x00007fa60e2042e6 in syncenv_processor (thdata=0x1995c30) at syncop.c:695
#3 0x00007fa60cc50dc5 in start_thread () from /lib64/libpthread.so.0
#4 0x00007fa60d87c21d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7fa60a15e700 (LWP 16214)):
#0 0x00007fa60cc54a82 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007fa60e204046 in syncenv_task (proc=0x1995870) at syncop.c:603
#2 0x00007fa60e2042e6 in syncenv_processor (thdata=0x1995870) at syncop.c:695
#3 0x00007fa60cc50dc5 in start_thread () from /lib64/libpthread.so.0
#4 0x00007fa60d87c21d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7fa60b55f700 (LWP 16213)):
#0 0x00007fa60cc54a82 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007fa60e204046 in syncenv_task (proc=0x1958400) at syncop.c:603
#2 0x00007fa60e2042e6 in syncenv_processor (thdata=0x1958400) at syncop.c:695
#3 0x00007fa60cc50dc5 in start_thread () from /lib64/libpthread.so.0
#4 0x00007fa60d87c21d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7fa60bd60700 (LWP 16212)):
#0 0x00007fa60cc54a82 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007fa60e204046 in syncenv_task (proc=0x1958040) at syncop.c:603
#2 0x00007fa60e2042e6 in syncenv_processor (thdata=0x1958040) at syncop.c:695
#3 0x00007fa60cc50dc5 in start_thread () from /lib64/libpthread.so.0
#4 0x00007fa60d87c21d in clone () from /lib64/libc.so.6
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel
