Multithreaded Iterative Dir Tree Scan

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello everyone!
Following a discussion I had with @Susant Palai some time ago, we have decided to look into an option to improve the rebalance process in the DHT layer by modifying the underlying mechanism. Currently, dir-tree crawling is done recursively, by a single thread, which is likely slow and also poses the risk of stack overflow. An iterative multithreaded solution might improve performance and also stability (by eliminating the risk of stack overflow). I have prepared a POC doc on the matter, including a sample implementation of the iterative multithreaded solution. The doc can be found at:
https://docs.google.com/document/d/1JCl0T9zeagOcFFpgVQF8zNyhlR54VqkNAZ7TJb42egE/edit
Apart from the rebalance process, maybe this approach can be useful for other use-cases where dir-tree crawl is being performed? Any comments on the concept, the design of the solution and the implementation are welcome.

--
Barak Sason Rofman

Gluster Storage Development

Red Hat Israel

34 Jerusalem rd. Ra'anana, 43501

bsasonro@redhat.com    T: +972-9-7692304
M: +972-52-4326355

_______________________________________________

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-devel


[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux