On Thu, Jun 06, 2013 at 10:25:53PM -0700, Adam Brenner wrote: > >> We run a 14TB XFS fileserver on our cluster and want to add quota > >> support. This is running 3.9.2-1.el6.elrepo.x86_64 kernel under CentOS > >> 6.3 (Final). > > > > How many files in the filesystem? > > 8,339,620 files. 11TB used / 14TB total. 8MB Used / 11GB inodes. > > (on a different server below) > 8,878,507 files. 2.1TB used / 5.5TB total. 9.3MB Used / 1.1GB inodes. > > Not that many compared to our larger file systems (already have quotas > enabled). But still a lot of files. Not excessive, but lets have a quick think about. Say there's some sparseness, so 25 allocated inodes to cluster, 200 IOs a second give 5,000 inodes a second, 300,000 inodes a minute, or roughly half an hour to scan the whole lot. I've got a filesystem with 50m inodes sitting on it right now, and quotacheck is running at about 1200 IOPS which is giving 27,000 inodes/s (on SSDs). That's about 1.5m inodes/min, so it should complete in 15-20min. It'd be faster, but there's another 3 VMs banging on the same SSDs running xfstests.... So you shouldn't be CPU bound at all, and speed will be determined by the IOPS that can be sustained by your hardware. > > You can't. quotacheck relies on exclusive access to the filesystem, > > and so the mount cannot process until it completes. Quotas need to > > be consistent before you can start using the filesystem.... > > > > Now, there's the possibility that we could parallelise quotacheck > > because it uses bulkstat for iteration, but somebody would have to > > find the time to do that and it won't solve your immediate problem. > > > > Hence I suggest just waiting for it to complete - if you have > > millions of inodes it could hours to complete, so just let it go > > overnight... > > Okay. Thanks for the information Dave! The servers we have are > critical in that having downtime causes a lot of unhappy users. Since > the XFS mount point is not accessible, everything that depends on it > needs to be taken offline -- for example users's home directories. > > I was trying to research a way to avoid such a thing, but doesn't seem > like I can. I would be interested to know, and maybe others on the > listsrv can chime in, on what sort of time period I can expect for the > quota check to run with 8 million files. Performing some martian math > I get something like 4 hours? Maybe I should switch planets :-) See my previous email about bstat. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
