Re: safer git?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



[I somehow didn't get the original message, so replying inline below.]

On 2020-10-24 at 22:11:54, Randall S. Becker wrote:
> On October 24, 2020 4:19 PM, hv@xxxxxxxxx wrote:
> > Q: Is there a mode in which I can run git that would make it a bit
> > more robust
> > against crashes, at the cost of being a bit slower?
> > 
> > 
> > The primary symptom is that files modified shortly before a crash
> > show up existing but zero-length after the crash. For source files I
> > mostly know what to do in that situation, but `git fsck` shows a lot
> > of files under '.git/objects' that are empty, which seems to make
> > things hard to recover:
> > 
> > % git fsck
> > error: object file
> > .git/objects/0e/f31631726cea2e9bf89d7bbe7b924b5282d533 is empty
> > error: unable to mmap
> > .git/objects/0e/f31631726cea2e9bf89d7bbe7b924b5282d533: No such file
> > or directory
> > error: 0ef31631726cea2e9bf89d7bbe7b924b5282d533: object corrupt or
> > missing: .git/objects/0e/f31631726cea2e9bf89d7bbe7b924b5282d533
> > [... a dozen similar entries ...]
> > error: object file
> > .git/objects/f5/a9d125645e69a0e40f9bf7a8c90b1c1c4a4ea5 is empty
> > error: unable to mmap
> > .git/objects/f5/a9d125645e69a0e40f9bf7a8c90b1c1c4a4ea5: No such file or
> > directory
> > error: f5a9d125645e69a0e40f9bf7a8c90b1c1c4a4ea5: object corrupt or
> > missing: .git/objects/f5/a9d125645e69a0e40f9bf7a8c90b1c1c4a4ea5
> > Checking object directories: 100% (256/256), done.
> > Checking objects: 100% (1577/1577), done.
> > error: refs/stash: invalid sha1 pointer
> > 0000000000000000000000000000000000000000
> > error: bad ref for .git/logs/refs/stash
> > dangling commit 1c0ea4e6159952501957012d2b9db7d68b52d107
> > %

You can try setting core.fsyncObjectFiles to true.  That's the only knob
that Git has for that at the moment, although there was some discussion
about adding a new feature for other files[0].

I suspect a lot of the zero-byte files and any files that end up as
all-zeros are due to your file system.  The default file system on
Ubuntu is ext4, IIRC, and if that's what you're using, you can set
data=journal instead of data=ordered as a mount option.  For the root
file system, you'll need to pass rootflags=data=journal as a kernel
boot option.

That may be significantly slower, but until you get your hardware
problem sorted out, it may very well be worth it for you.  I'd try this
option before the one below because it'll have less of an impact on
performance and may solve most or all of your problems.

> > Last time I checked out the previous state from github in a new directory
> and
> > was able to find and copy over most of my work before continuing. On this
> > occasion I did a `git stash save` shortly before the crash, and I'm not
> sure
> > how to get that back. I see René Scharfe's suggestion of:
> >   git fsck --unreachable |
> >   grep commit | cut -d\  -f3 |
> >   xargs git log --merges --no-walk --grep=WIP from a recent message, but
> > that is only showing me an older stash item.
> 
> I would suggest turning off write-through buffering on your disk. Let writes
> complete immediately instead of being deferred to sync. Also, this does feel
> like a disk issue, so fsck or chkdsk /f (or whatever) on your disk urgently.

Turning off buffering and caching for your disk drive may make things
_really_ slow, but it will definitely improve data integrity.

I know hardware problems are always a hassle, so I hope you get things
figured out and fixed soon.

[0] I admit I am not running the very latest version and the new feature
may have already landed; if so, I apologize for the out-of-date
information and for not keeping up with the list.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

Attachment: signature.asc
Description: PGP signature


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux