On 10 November 2012 01:43, Atchley, Scott <atchleyes@xxxxxxxx> wrote: > On Nov 8, 2012, at 5:00 PM, Joseph Glanville <joseph.glanville@xxxxxxxxxxxxxx> wrote: > >> On 9 November 2012 08:21, Dieter Kasper <d.kasper@xxxxxxxxxxxx> wrote: >>> Joseph, >>> >>> I've downloaded and read the presentation from 'Sean Hefty / Intel Corporation' >>> about rsockets, which sounds very promising to me. >>> Can you please teach me how to get access to the rsockets source ? >>> >>> Thanks, >>> -Dieter >>> >>> >> >> rsockets is distributed as part of librdmacm. You can clone the git >> repository here: >> git://beany.openfabrics.org/~shefty/librdmacm.git >> >> I recommend using the latest master as it features much better support >> for forking. > > I would be interested in hearing about how it works at scale. I do not know if Sean uses dedicated send and receive buffers per connection or a shared receive queue or shared send queue. Scaling might be an issue or it might not. > > Scott Hi Scott, It uses RC QPs so it's scalability is as good as any other large scale app with similar HCA resource requirements. As an aside: As I understand it Mellanox is working on a new piece of hardware called Connect-IB (somewhat of a successor to Connext-X 3) that will use a temporary/virtual resource mapping for RC QPs that should make this a non-issue. Press release is here: http://www.mellanox.com/content/pages.php?pg=press_release_item&rec_id=814 The HCA is just in general much beefier in terms of available resources for QPs and MRs so it's a big boon for big clusters than need to use all-to-all RC QPs. That being said I don't have a computer with enough nodes for it to make a difference. :( Joseph. -- CTO | Orion Virtualisation Solutions | www.orionvm.com.au Phone: 1300 56 99 52 | Mobile: 0428 754 846 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
