Thanks Kefu. I'll write code based on your suggestions. -----Original Message----- From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of kefu chai Sent: Monday, August 27, 2018 7:35 PM To: The Esoteric Order of the Squid Cybernetic <ceph-devel@xxxxxxxxxxxxxxx> Cc: Cheng, Yingxin <yingxin.cheng@xxxxxxxxx>; Ma, Jianpeng <jianpeng.ma@xxxxxxxxx>; Liu, Chunmei <chunmei.liu@xxxxxxxxx> Subject: Re: How to implement OSDMap cache for Seastar? + ceph-devel ----- Original Message ----- > From: "Jianpeng Ma" <jianpeng.ma@xxxxxxxxx> > To: "Kefu Chai" <kchai@xxxxxxxxxx> > Cc: "Yingxin Cheng" <yingxin.cheng@xxxxxxxxx>, "Chunmei Liu" > <chunmei.liu@xxxxxxxxx> > Sent: Monday, August 27, 2018 4:49:48 PM > Subject: How to implement OSDMap cache for Seastar? > > Hi kefu: > Based on those : > a)https://github.com/ceph/ceph/projects/2 > b) https://www.spinics.net/lists/ceph-devel/msg41613.html > > * How to remove lock in SharedLRU? > Because osdmap from peer osds and monitor, so if remove lock in > SharedLRU, we should make a owner-thread do osdmap update, > etc i think the shared osdmap will be a mini sharded-service in OSD. and the only a single shard will be managing the cache, the other shards will be wrap the pointers as foreign_ptrs<> before passing them to/from the "owner" shard. as you put, the cached osdmaps are contributed from its peer osds and monitor. i think an option is to allocate the osdmap and their encoded form only if they are read from a "local" connection or from the pg hosted by this shard. > choose the core of osd-monitor connection do this. > * Who allocate OSDMap and who free OSDMap? > Currently for SharedLRU, it use template<> > struct SharedPtrTrait<LockPolicy::MUTEX> { > template<class T> using shared_ptr = std::shared_ptr<T>; > template<class T> using weak_ptr = std::weak_ptr<T>; > }; > In a<https://github.com/ceph/ceph/projects/2>, you used > seastar::foreign_ptr<> and boost::local_shared_ptr. > struct SeaStarSharedPtrTrait<> { > template<class T> using shared_ptr = seastar∷foreign_ptr<T>; > template<class T> using weak_ptr = boost::local_shared_ptr <T>; > }; > > For weak_ptr, why does not use seastar∷lw_shared_ptr? please note, weak_refs does not *own" maps. while seastar::lw_shared_ptr holds a strong reference of the managed pointer. and seastar::weak_ptr is not able to promote a weak_ptr to shared_ptr or lw_shared_ptr. > > And for typedef std::shared_ptr<const OSDMap> OSDMapRef; > We can change to: typedef > seastar∷foreign_ptr<seastar∷lw_shared_ptr<const OSDMap > > OSDMapRef. > In a core, we still need a smart point for OSDMap, etc > seastart∷shared_ptr or seastar∷lw_shared_ptr. seastar::lw_shared_ptr would suffice. if we don't need polymorphism. > And we need multi core share OSDMap, so we use seastar∷foreign_pt to > make the same core allocate and free memory. > agreed. > > Thanks! > Jianpeng > > > > >
