On Sat, Jan 10, 2009 at 9:54 PM, Manish Katiyar <mkatiyar@xxxxxxxxx> wrote: > On Sat, Jan 10, 2009 at 9:49 PM, Manish Katiyar <mkatiyar@xxxxxxxxx> wrote: >> On Sat, Jan 10, 2009 at 9:09 PM, Sandeep K Sinha >> <sandeepksinha@xxxxxxxxx> wrote: >>> See the problem here would be that the inode number would change in >>> case we allocate a new inode for our destination file after >>> relocation. >> >> One basic question that should have been probably asked long back >> *unless i missed if you answered it*. What is the pathname for your >> new destination inode ? Is it same as old path ? Though my guess says >> it should be same as the old path. >> >> Coming back to the hard link problem, yes the inode number will change >> once you allocate a new inode and then one of the *clean* but poor >> solution is to go and update all the dentries in the filesystem, or >> another "dirty* but easy way is to just convert the old inode into a >> symlink to the old path and everything should work seamlessly. Of >> course you have added one level of redirection and you might run out >> of inodes after you have relocated many files. Possibly a kernel > > BTW if running out of inodes is really not that big of an issue, then > I kind of prefer the idea of creating a new pathname and convert the > old inode as symlink to the new one. Argument is that this would help > aministrators to track and validate how much efficient fscops is. So > if the old file is > > /home/myfile (inum1) ... move it to /fscops_relocated/home/myfile > (inum2) and let /home/myfile be a symlink to the new file. > Agreed, but tell me any good reason why I should allocate a new inode ? Don't you think updating the data blocks in the old inode itself, suffice our intentions. The real cost of i/o would be in reading/writing data blocks and not inode. So, even if the inode remains allocated on the old tier, its data blocks would be actually allocated from a different tier. Which should provide all that we need. > Thanks - > Manish > > >> thread which fixes them again in background so your relocation happens >> something like this :- >> >> a) Allocate new inode (inum2) >> b) Swap pointers from inum1 to inum2 and write them to disk. >> c) Convert inum1 to symlink to inum2 >> d) Let the background kernel thread try to go and fix the >> relationships. So somewhere in between when you have not fixed all of >> them, you will have some hardlinks to inum2 and some hardlinks to >> inum1. >> >> Does that make sense at all ? >> >> thanks - >> Manish >> >> >>> >>> How will we update the inode number in the hard links ? >>> >>> On Sat, Jan 10, 2009 at 8:31 PM, Manish Katiyar <mkatiyar@xxxxxxxxx> wrote: >>>> On Sat, Jan 10, 2009 at 1:38 PM, Sandeep K Sinha >>>> <sandeepksinha@xxxxxxxxx> wrote: >>>>> Hi Greg, >>>>> >>>>> On Fri, Jan 9, 2009 at 5:49 AM, Greg Freemyer <greg.freemyer@xxxxxxxxx> wrote: >>>>>> Both a top post and bottom post. >>>>>> >>>>>> == Top Post >>>>>> >>>>>> Lost the context for this, but looking at your site: >>>>>> http://code.google.com/p/fscops/ >>>>>> >>>>>> I see a very valuable HSM goal, but I don't see the biggest user of >>>>>> future HSM implementations. >>>>>> >>>>> Do you mean the futuristic aspects of this project in the market or >>>>> something to be mentioned on the project webpage. >>>>> >>>>>> Namely servers that add SSD drives as even faster / more expensive >>>>>> storage than rotating disks. >>>>>> >>>>> >>>>> >>>>> >>>>>> The block layer is right now is exposing a flag to user space for >>>>>> rotating / non-rotating. I think it will be in 2.6.29, but it has not >>>>>> been accepted yet I don't think. >>>>>> >>>>>> Since non-rotating media is about 50x the cost of rotating media, I >>>>>> can envision a lot of useres wanting to put fiels that are about to be >>>>>> heavily used onto SSD for processing / searching / etc. then moved >>>>>> back to rotating disk until needed again. >>>>>> >>>>>> I know my companies data usage patterns would benefit greatly from an >>>>>> HSM that support SSD drives. We work with a couple hundred datasets a >>>>>> year typically. But at any given time, just a couple of them are >>>>>> actively in use. >>>>>> >>>>> >>>>> Thanks for your insight. Well, if I am getting it right, does that >>>>> mean that such utilities will be of great use in future ? >>>>> We already have several use cases for such tiered storage. Database >>>>> environment and search engines being one of the major ones. >>>>> >>>>>>>> copy_inode_info() >>>>>>> >>>>>>> Well, currently I dont intend to move the inode to a new location. I >>>>>>> would prefer leave the original inode intact just updating the new >>>>>>> data block pointers. This is still in debate, whether to relocate >>>>>>> inode or not. >>>>>> >>>>>> I know I pseudo coded it up based on a new inode. >>>>>> >>>>>> If your goal is to do the HSM re-org with live files, then I would try >>>>>> real hard not to allocate a new inode. >>>>>> >>>>> >>>>> Allocating a new inode will also cause several issues for hardlinks as well. >>>>> Manish, does ext2 have support for hardlinks, any idea ? >>>> >>>> Well.... I don't understand the problem here. Can you be a bit more >>>> specific. What do you mean by "ext2 have support for hardlinks" . Also >>>> how the problem you are trying to solve is going to be solved by >>>> hardlinks. >>>> >>>> >>>> >>>>> >>>>>> That way after the re-org all of the file i/o will continue to work >>>>>> exactly like it was. >>>>>> >>>>> What if the file which is being copied is of several TB's we will have >>>>> to allocate a ghost inode of the same size, which might cause two disk >>>>> space on the file system for sometime, failing some of the >>>>> applications which tend to allocated/need large number of data blocks. >>>>> Is there any way to avoid such situations ? >>>>> >>>>>> Greg >>>>>> -- >>>>>> Greg Freemyer >>>>>> Litigation Triage Solutions Specialist >>>>>> http://www.linkedin.com/in/gregfreemyer >>>>>> First 99 Days Litigation White Paper - >>>>>> http://www.norcrossgroup.com/forms/whitepapers/99%20Days%20whitepaper.pdf >>>>>> >>>>>> The Norcross Group >>>>>> The Intersection of Evidence & Technology >>>>>> http://www.norcrossgroup.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Sandeep. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> "To learn is to change. Education is a process that changes the learner." >>>>> >>>> >>> >>> >>> >>> -- >>> Regards, >>> Sandeep. >>> >>> >>> >>> >>> >>> >>> "To learn is to change. Education is a process that changes the learner." >>> >> > -- Regards, Sandeep. "To learn is to change. Education is a process that changes the learner." -- To unsubscribe from this list: send an email with "unsubscribe kernelnewbies" to ecartis@xxxxxxxxxxxx Please read the FAQ at http://kernelnewbies.org/FAQ
