> > On Wednesday, July 27, 2016, Vladislav Bolkhovitin <vst@xxxxxxxx> wrote: >> >> >> Alex Gorbachev wrote on 07/27/2016 10:33 AM: >> > One other experiment: just running blkdiscard against the RBD block >> > device completely clears it, to the point where the rbd-diff method >> > reports 0 blocks utilized. So to summarize: >> > >> > - ESXi sending UNMAP via SCST does not seem to release storage from >> > RBD (BLOCKIO handler that is supposed to work with UNMAP) >> > >> > - blkdiscard does release the space >> >> How did you run blkdiscard? It might be that blkdiscard discarded big >> areas, while ESXi >> sending UNMAP commands for areas smaller, than min size, which could be >> discarded, or >> not aligned as needed, so those discard requests just ignored. Here is the output of the debug, many more of these statements before and after. Is it correct to state then that SCST is indeed executing the discard and the RBD device is ignoring it (since the used size in ceph is not diminishing)? Jul 30 21:08:46 e1 kernel: [ 3032.199972] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570716160, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.202622] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570724352, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.207214] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570732544, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.210395] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570740736, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.212951] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570748928, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.216187] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570757120, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.219299] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570765312, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.222658] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570773504, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.225948] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570781696, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.230092] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570789888, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.234153] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570798080, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.238001] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570806272, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.240876] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570814464, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.242771] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570822656, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.244943] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570830848, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.247506] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570839040, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.250090] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570847232, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.253229] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570855424, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.256001] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570863616, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.259204] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570871808, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.261368] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570880000, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.264025] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570888192, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.266737] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570896384, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.270143] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570904576, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.273975] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570912768, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.278163] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570920960, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.282250] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570929152, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.285932] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570937344, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.289736] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570945536, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.292506] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570953728, nr_sects 8192) Jul 30 21:08:46 e1 kernel: [ 3032.294706] [22016]: vdisk_unmap_range:3830:Discarding (start_sector 570961920, nr_sects 8192) Thank you, Alex > > > I indeed ran blkdiscard on the whole device. So the question to the Ceph > list is below what length discard is ignored? I saw at least one other user > post a similar issue with ESXi-SCST-RBD. > >> >> >> For completely correct test you need to run blkdiscard for exactly the >> same areas, both >> start and size, as the ESXi UNMAP requests you are seeing on the SCST >> traces. > > > I am running a test with the debug settings you provided, and will keep this > thread updated with results. Much appreciate the guidance. > > Alex > >> >> >> Vlad >> > > > -- > -- > Alex Gorbachev > Storcium > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |