On Wed, Jun 11, 2008 at 05:34:53PM +0000, Linus Torvalds wrote: > > > > > > Quite often, when people commit, they have corrupt repositories. The > > > > symptom is a `cannot read <sha1>` error message (or many at times). > > Btw, do you have exact error messages? Well, like said I don't have had the problem myself, and it's hard to reproduce, people have it like once a week average, for people that perform around 50 to 100 commits a week. > That plain "cannot read <sha1>" sounds unlikely. It exists in archive-tar, > archive-zip and builtin-tag (and in two of the tests), but not in any > commit paths that I can tell. Well that was what I recalled, I'll pass the word that I want a copy of the message so that people don't trash their history when it happens. On Wed, Jun 11, 2008 at 05:25:28PM +0000, Linus Torvalds wrote: > Do you have people using special flags for your NFS mounts? And do you > know if there is some pattern to the client kernel versions when the > problem happens? not that I'm aware of. One machine with issues has this: 192.168.2.2:/home on /home type nfs (rw,noatime,bg,intr,hard,tcp,rsize=65536,wsize=65536,addr=192.168.2.2) IOW nothing fancy that I can see. On Wed, Jun 11, 2008 at 05:46:00PM +0000, Linus Torvalds wrote: > On Wed, 11 Jun 2008, Linus Torvalds wrote: > > > > Do you have people using special flags for your NFS mounts? And do you > > know if there is some pattern to the client kernel versions when the > > problem happens? > > Oh, before I even go there - let's get the _really_ obvious case out of > the way first. > > If you are using a shared git object repository (why are you doing that, > btw?), are people perhaps doing things like "git gc --auto" etc at the > same time? Perhaps even unknowingly, thanks to autogc? No, we're not using a shared git object repository, each developper has a git checkout in his /home (on NFS) but works for real in a workdir that lives on his local hard drive (to get faster compilation times, because NFS really sucks at speed for compilation). Though, people working on plain NFS have had the same problems. I also had the same reaction as you do, and I've seen such problems occur, and the operation that we were doing was just "git commit -as" or something very similar. No nothing in progress in any other terminal at the same time. > So if one client is doing some kind of gc and creates a new pack-file and > then removes old loose objects, and another client has already looked up > and opened that loose object (but not finished reading it), then when the > file gets removed, you will literally lose the data on the other client > and get a short read! I don't think that is what happens. > And nothing we can do can ever fix this very fundamental issue of NFS. NFS > simply isn't an even remotely POSIX filesystem, even though it's set up to > mostly _look_ like one when accessed from a single client. > > In general, I would discourage people ever sharing object directories > among multiple users except in a server kind of environment (eg > kernel.org). It's not shared repositories, really, it's just for conveniency so that the object store is on NFS (and backuped like all that is on the NFS server) but the checkouts on local hard drives, so that all operations to the checkout are local. And when people commit, it goes into the NFS and all is perfectly fine. Those repositories are *ONLY* used by one physical user exclusively. I'll try to do some commits on NFS repeatedly tonight and trigger the issue, I'll keep you posted. -- ·O· Pierre Habouzit ··O madcoder@xxxxxxxxxx OOO http://www.madism.org
Attachment:
pgpynbu9StSsL.pgp
Description: PGP signature
