First forgive me for using outlook for this, if there are any issues with what I sent let me know and I'll send it again from gmail. This is also my first attempt at a kernel patch so please be gentle. This patch was written to enable tape statistics via sysfs for the dt driver based on kernel 3.8.0-rc6. It creates two new files in sysfs and is based on work done previously in 2005 by Kai Mäkisara. Any feedback would be greatly appreciated. Assuming sysfs is mounted at /sys the first file is /sys/bus/scsi/drivers/st/drives which gives a single number indicating what the largest tape drive instance assigned by st_probe in the st module is. If it's 4 it possible that st0, st1, st2, and st3 exist on the system. Since tape drives can later be disconnected they don't have to exist, the count is a hint so it's possible to gather statistics in a loop with an upper bound. This makes it easier in iostat to gather statistcs. The second file is /sys/class/scsi_tape/stxx/stat where xx is the instance of the tape drive. The file contents are almost the same as the stat file for disks except the merge statistics are always 0 (since tape drives are sequential merged I/Os don't make sense) and the inflight value is either a 0 or 1 since the st module always only has either one read or write outstanding. I've also added one field to the end of the file - a count other I/Os - this could be commands issued by the driver within the kernel (e.g. rewind) or via an ioctl from user space. For tape drives some commands involving actions like tape movement can take a long time, it's important to keep track of scsi requests sent to the tape drive other than reads and writes so when delays happen they can be explained. With some future patches to iostat this figure will be reported and used to calculate an average wait for all I/Os (a_await and oio/s in this output): tape: wr/s KiB_write/s rd/s KiB_read/s r_await w_await a_await oio/s st0 186.50 46.75 0.00 0.00 0.000 0.276 0.276 0.00 st1 186.00 93.00 0.00 0.00 0.000 0.180 0.180 0.00 st2 0.00 0.00 181.50 45.50 0.347 0.000 0.347 0.00 st3 0.00 0.00 183.00 45.75 0.224 0.000 0.224 0.00 Q: Does anyone have strong objections to extending the stat format to include another field (a count of scsi commands issue to the target other than reads or writes), or should the format stay in common with disks and a new device class specific file be created that provides extra statistics that may be useful only for a specific class of SCSI device? For example called stat-tape, stat-st or something else? Onto justification we have a customer using virtual tape libraries (lots of drives) and they wanted to be able to monitor the activity and performance of their backups. Because of a lack of functionality they resorted to using a publicly available SystemTap script (created by RedHat presumably when they received similar requests from other customers): http://sourceware.org/systemtap/wiki/WSiostatSCSI Unfortunately, using this script occasionally results in kernel panics on older kernels, those issues have been addressed but most customers still don't end up running the SystemTap script unless they have to and they still wait to monitor performance of their tape drives. Just googling: linux tape throughput statistics is enough to yield many hits on the topic including these: 1. http://www.ibm.com/developerworks/forums/thread.jspa?messageID=14775056 2. http://h30499.www3.hp.com/t5/System-Administration/How-to-get-tape-drive-performance-stats/td-p/3880235#.UKoJxNGloUo 3. http://docs.oracle.com/cd/E19455-01/816-3319/6m9k06r58/index.html The first two are asking about getting tape stats on Linux, the reply for 1. is that you can get the information on AIX. 2. is similar but the reply is that you can get the information for HP-UX 11.31. The last one is the iostat manual page for Solaris which can report tape stats as well. All 3 point out that iostat can print tape statistics on the largest of the commercial unix operating systems. Q: Does anyone have any general feedback about things that need to change or demands about changing the implementation before being accepted? The checkpatch.pl script generates warnings for the diffs because of CamelToe however the CamelToe warnings are because I wanted to stay consistent with the module (look for things like STp). Signed-off-by: Shane Seymour <shane.seymour@xxxxxx> Signed-off-by: Darren Lavender <dcl@xxxxxxxxxxxxxxxxxxx> Tested-by: Shane Seymour <shane.seymour@xxxxxx> Tested-by: Darren Lavender <dcl@xxxxxxxxxxxxxxxxxxx> --- diff -uprN -X linux-3.8-rc6-vanilla/Documentation/dontdiff linux-3.8-rc6-vanilla/drivers/scsi/st.c linux-3.8-rc6/drivers/scsi/st.c --- linux-3.8-rc6-vanilla/drivers/scsi/st.c 2013-02-08 14:35:27.000000000 +0000 +++ linux-3.8-rc6/drivers/scsi/st.c 2013-02-22 00:06:50.000000000 +0000 @@ -174,6 +174,9 @@ static int debugging = DEBUG; static int st_fixed_buffer_size = ST_FIXED_BUFFER_SIZE; static int st_max_sg_segs = ST_MAX_SG; +/* This is the highest drive number plus one */ +static int st_last_drive; + static int modes_defined; static int enlarge_buffer(struct st_buffer *, int, int); @@ -458,10 +461,19 @@ static void st_scsi_execute_end(struct r struct st_request *SRpnt = req->end_io_data; struct scsi_tape *STp = SRpnt->stp; struct bio *tmp; + u64 ticks = get_jiffies_64(); STp->buffer->cmdstat.midlevel_result = SRpnt->result = req->errors; STp->buffer->cmdstat.residual = req->resid_len; + STp->in_flight--; + ticks -= STp->stamp; + STp->io_ticks += ticks; + if (req->cmd[0] == WRITE_6) + STp->write_ticks += ticks; + else if (req->cmd[0] == READ_6) + STp->read_ticks += ticks; + tmp = SRpnt->bio; if (SRpnt->waiting) complete(SRpnt->waiting); @@ -478,6 +490,7 @@ static int st_scsi_execute(struct st_req struct rq_map_data *mdata = &SRpnt->stp->buffer->map_data; int err = 0; int write = (data_direction == DMA_TO_DEVICE); + struct scsi_tape *STp = SRpnt->stp; req = blk_get_request(SRpnt->stp->device->request_queue, write, GFP_KERNEL); @@ -498,6 +511,18 @@ static int st_scsi_execute(struct st_req } } + if (cmd[0] == WRITE_6) { + STp->write_cnt++; + STp->write_byte_cnt += bufflen; + } else if (cmd[0] == READ_6) { + STp->read_cnt++; + STp->read_byte_cnt += bufflen; + } else { + STp->other_cnt++; + } + STp->stamp = get_jiffies_64(); + STp->in_flight++; + SRpnt->bio = req->bio; req->cmd_len = COMMAND_SIZE(cmd[0]); memset(req->cmd, 0, BLK_MAX_CDB); @@ -1241,6 +1266,16 @@ static int st_open(struct inode *inode, } STp->try_dio_now = STp->try_dio; STp->recover_count = 0; + STp->read_byte_cnt = 0; + STp->write_byte_cnt = 0; + STp->read_cnt = 0; + STp->write_cnt = 0; + STp->other_cnt = 0; + STp->in_flight = 0; + STp->read_ticks = 0; + STp->write_ticks = 0; + STp->io_ticks = 0; + DEB( STp->nbr_waits = STp->nbr_finished = 0; STp->nbr_requests = STp->nbr_dio = STp->nbr_pages = 0; ) @@ -4123,6 +4158,17 @@ static int st_probe(struct device *dev) tpnt->buffer = buffer; tpnt->buffer->last_SRpnt = NULL; + tpnt->read_byte_cnt = 0; + tpnt->write_byte_cnt = 0; + tpnt->read_cnt = 0; + tpnt->write_cnt = 0; + tpnt->other_cnt = 0; + tpnt->in_flight = 0; + tpnt->read_ticks = 0; + tpnt->write_ticks = 0; + tpnt->io_ticks = 0; + tpnt->stamp = 0; + tpnt->inited = 0; tpnt->dirty = 0; tpnt->in_use = 0; @@ -4208,6 +4254,13 @@ static int st_probe(struct device *dev) goto out_remove_devs; scsi_autopm_put_device(SDp); +/* If there are no tape drives or one leaving st_last_drive at 0 would be + confusing when read from user space so it contains the highest drive + number plus one - zero means no drives controlled by this driver. */ + + if (dev_num+1 > st_last_drive) + st_last_drive = dev_num+1; + sdev_printk(KERN_NOTICE, SDp, "Attached scsi tape %s\n", tape_name(tpnt)); sdev_printk(KERN_INFO, SDp, "%s: try direct i/o: %s (alignment %d B)\n", @@ -4364,6 +4417,12 @@ static ssize_t st_version_show(struct de } static DRIVER_ATTR(version, S_IRUGO, st_version_show, NULL); +static ssize_t st_drives_show(struct device_driver *ddp, char *buf) +{ + return snprintf(buf, PAGE_SIZE, "%d\n", st_last_drive); +} +static DRIVER_ATTR(drives, S_IRUGO, st_drives_show, NULL); + static int do_create_sysfs_files(void) { struct device_driver *sysfs = &st_template.gendrv; @@ -4381,9 +4440,14 @@ static int do_create_sysfs_files(void) err = driver_create_file(sysfs, &driver_attr_version); if (err) goto err_attr_max_sg; + err = driver_create_file(sysfs, &driver_attr_drives); + if (err) + goto err_attr_drives; return 0; +err_attr_drives: + driver_remove_file(sysfs, &driver_attr_drives); err_attr_max_sg: driver_remove_file(sysfs, &driver_attr_max_sg_segs); err_attr_fixed_buf: @@ -4397,6 +4461,7 @@ static void do_remove_sysfs_files(void) { struct device_driver *sysfs = &st_template.gendrv; + driver_remove_file(sysfs, &driver_attr_drives); driver_remove_file(sysfs, &driver_attr_version); driver_remove_file(sysfs, &driver_attr_max_sg_segs); driver_remove_file(sysfs, &driver_attr_fixed_buffer_size); @@ -4478,12 +4543,30 @@ options_show(struct device *dev, struct return l; } +static ssize_t +stat_show(struct device *dev, struct device_attribute *attr, char *buf) +{ + struct st_modedef *STm = dev_get_drvdata(dev); + struct scsi_tape *STp = STm->tape; + +/* io_ticks includes time spent doing internal commands that are required + for the tape driver to work like rewinding the tape drive for a + rewind device. */ + return snprintf(buf, PAGE_SIZE, "%llu 0 %llu %u %llu 0 %llu %u %llu %u %u %llu", + STp->read_cnt, STp->read_byte_cnt >> 9, + jiffies_to_msecs(STp->read_ticks), STp->write_cnt, + STp->write_byte_cnt >> 9, jiffies_to_msecs(STp->write_ticks), + STp->in_flight, jiffies_to_msecs(STp->io_ticks), + jiffies_to_msecs(STp->io_ticks), STp->other_cnt); +} + static struct device_attribute st_dev_attrs[] = { __ATTR_RO(defined), __ATTR_RO(default_blksize), __ATTR_RO(default_density), __ATTR_RO(default_compression), __ATTR_RO(options), + __ATTR_RO(stat), __ATTR_NULL, }; diff -uprN -X linux-3.8-rc6-vanilla/Documentation/dontdiff linux-3.8-rc6-vanilla/drivers/scsi/st.h linux-3.8-rc6/drivers/scsi/st.h --- linux-3.8-rc6-vanilla/drivers/scsi/st.h 2013-02-08 14:35:18.000000000 +0000 +++ linux-3.8-rc6/drivers/scsi/st.h 2013-02-22 00:08:42.000000000 +0000 @@ -158,6 +158,19 @@ struct scsi_tape { int max_block; int recover_count; /* From tape opening */ int recover_reg; /* From last status call */ +/* Tape stats */ + u64 read_byte_cnt; /* bytes read since tape open */ + u64 write_byte_cnt; /* bytes written since tape open */ + u64 in_flight; /* Number of I/Os in flight */ + u64 read_cnt; /* Count of read requests since tape open */ + u64 write_cnt; /* Count of write requests since tape open */ + u64 other_cnt; /* Count of other requests since tape open + either implicit (from driver) or from + user space via ioctl. */ + u64 read_ticks; /* Ticks spent completing read requests */ + u64 write_ticks; /* Ticks spent completing write requests */ + u64 io_ticks; /* Ticks spent doing any I/O */ + u64 stamp; /* holds time request was queued */ #if DEBUG unsigned char write_pending; -- To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
