On Thu, May 12, 2016 at 1:37 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > On Thu, 12 May 2016, Naveen wrote: >> Sage Weil <sage <at> newdream.net> writes: >> >> >> > > > My question is: >> > > > The SCSI LLD would support both read /write entry points for I/O >> > > > requests issued by the filesystem/block I/O but they also support >> some >> > > > custom requests using IOCTLs. So how can ceph support issuing of >> such >> > > > IOCTL requests to the device if user issues such request. Say for >> > > > example power cycling the drive etc. It can also be a passthro >> request >> > > > down to the device. >> > >> > Can you give an example of such an operation? >> > >> > In general, any operation is generalized at the librados level. >> > For example, in order to get write same and cmpxchg block operations, >> we >> > added librados operations with similar semantics and implement them >> there. >> > It is unlikely that passing a complex operation down to the SCSI layer >> > will work in unison with the other steps involved in committing >> > an operation (e.g., updating metadata indicating the object >> > version has changed). >> > >> > sage >> > >> > -- >> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" >> in >> > the body of a message to majordomo <at> vger.kernel.org >> > More majordomo info at http://vger.kernel.org/majordomo-info.html >> > >> > >> >> Thanks for the response Sage. Example IOCTL operations would be like >> downloading a new FW to the drive/HBA, task management requests like >> Hard reset, power cycling the drive, issue a SAS/SMP/STP pass thro >> command to the drive for querying etc. All these would have to be >> initiated and go through ceph (if supported) and not bypassing it. > > Ah. I think these kind of management functions should be performed while > the ceph-osd daemon for that drive is offline. We would probably want > some hardware management layer that coexists with ceph or that perhaps has > some minimal integration with the ceph osds to do this sort of thing. > It's not something that a client (user) would initiate, though. This is all pretty relevant to Joe Handzik's stuff: https://github.com/joehandzik/ceph/commits/wip-hw-mgmt-cli http://www.spinics.net/lists/ceph-devel/msg30126.html The idea there though is to enable passing libstoragemgmt calls through the OSD, as opposed to arbitrary SCSI operations. Although libstoragemgmt is fairly young, I'm a fan of the idea that we could use it internally within Ceph, and then have the same tools/libs used by out-of-ceph management platforms when they want to do equivalent stuff while the OSD is offline. John >> I asked a related question in another post too: Can a physical disk >> (/dev/sda1) assigned to a ceph OSD object be continued to used by other >> apps in the system to issue I/O directly via (/dev/sda1) interface? Does >> ceph prevent it, as such operations may corrupt data? > > It depends on what privileges the other app has. If it's root or user > ceph, it can step all over the disk (and the rest of the system) and wreak > havoc. With the current backend, we are storing data as files, so you > could have other apps using other directories on the file system--this is > generally a bad idea for real deployments, though, as performance and disk > utilization will be unpredictable. > > sage > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
