> From: Andrey <ahippo@xxxxxxxxx>
> Sent: Wednesday, 31 July, 2019 21:35
> To: Philip McGraw <Philip.McGraw@xxxxxxxxxxx>
> Cc: git@xxxxxxxxxxxxxxx; luke@xxxxxxxxxxx
> Subject: Re: [PATCH] git-p4: close temporary file before removing
>
> 31.07.2019, 17:52, "Philip McGraw" <philip.mcgraw@xxxxxxxxxxx>:
> > 2019.07.31 10:09 Andrey <ahippo@xxxxxxxxx>
> >> 31.07.2019, 09:53, "Philip McGraw" <philip.mcgraw@xxxxxxxxxxx>:
> >>>> 30.07.2019, 13:37, "Philip McGraw" <philip.mcgraw@xxxxxxxxxxx>:
> >>>> > python os.remove() throws exceptions on Windows platform when
> attempting
> >>>> > to remove file while it is still open. Need to grab filename
> while file open,
> >>>> > close file handle, then remove by name. Apparently other
> platforms are more
> >>>> > permissive of removing files while busy.
> >>>> > reference:
> >>>> > ---
> >>>> > git-p4.py | 4 +++-
> >>>> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >>>> >
> >>>> > diff --git a/git-p4.py b/git-p4.py
> >>>> > index c71a6832e2..6b9d2a8317 100755
> >>>> > --- a/git-p4.py
> >>>> > +++ b/git-p4.py
> >>>> > @@ -1161,12 +1161,14 @@ def exceedsLargeFileThreshold(self,
> relPath, contents):
> >>>> > return False
> >>>> > contentTempFile = self.generateTempFile(contents)
> >>>> > compressedContentFile =
> tempfile.NamedTemporaryFile(prefix='git-p4-large-file', delete=False)
> >>>> > + compressedContentFileName = compressedContentFile.name
> >>>> > zf = zipfile.ZipFile(compressedContentFile.name,
> mode='w')
> >>>> > zf.write(contentTempFile,
> compress_type=zipfile.ZIP_DEFLATED)
> >>>> > zf.close()
> >>>> > compressedContentsSize =
> zf.infolist()[0].compress_size
> >>>> > os.remove(contentTempFile)
> >>>> > - os.remove(compressedContentFile.name)
> >>>> > + compressedContentFile.close()
> >>>> > + os.remove(compressedContentFileName)
> >>>>
> >>>> I'm not sure why NamedTemporaryFile() is called with delete=False
> above,
> >>>> but it appears to me that it can have delete=True instead,
> >>>> so that there is no need to call os.remove() explicitly
> >>>> and thus worry about remove vs close ordering at all.
> >>>>
> >>>> > if compressedContentsSize > gitConfigInt('git-
> p4.largeFileCompressedThreshold'):
> >>>> > return True
> >>>> > return False
> >>>> > --
> >>>> > 2.21.0.windows.1
> >>>>
> >>>> Thank you,
> >>>> Andrey.
> >>>
> >>> Thanks Andrey; simpler is certainly better! I will test and re-submit
> v2 of patch with that approach.
> >>
> >> Thank you, that would be great!
> >>
> >> --
> >> Andrey.
> >
> > Unfortunately it wasn't as simple it seemed: upon testing with only
> changing delete=True,
> > found that the problem was not solved. Upon further debugging,
> recoded/refactored slightly adding
> > allocateTempFileName() locally scoped function to try to clarify how the
> NamedTemporaryFile()
> > was actually being used.
> >
> > We can't depend on the delete-on-close because the NamedTemporaryFile()
> is merely allocating
> > a temporary name for real use by the zipfile open-for-write which fails
> (on Windows) if file
> > was not explicitly closed first.
>
> Oh, sorry for misguiding you!
> I didn't think of this aspect.
No worries! I probably just misunderstood the implementation of your idea.
>
> > Hopefully the new patch
> (https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__github.com_gitgitgadget_git_pull_301&d=DwIDaQ&c=hmGTLOph1qd_VnCqj81HzE
> WkDaxmYdIWRBdoFggzhj8&r=b0ikFMJGw7xxhF3yjexiWJpLuNxlAh1SvUDuUJ-
> pHmE&m=1jGOrV_I1Mg5ajkJ7yFEcNlyLnD6zYNXqXB9Z5SIPyE&s=TdT4WHyQCk5WZty_CvajH
> XgrZJbmIOl1gbMcngmjmAs&e= ) will make this more clear.
>
> The new changeset looks good to me.
> (I'll post a reply in that thread too)
>
> > Open to other suggestions if still not clear.
>
> Just as a thought, ZipFile() can take a file-like object instead of a file
> name,
> so can be passed the NamedTemporaryFile() object directly instead of its
> file name.
> This should hopefully avoid double-open issue on Windows.
Another excellent idea that minimizes changes. I am testing this approach
now and will submit v3 of the patch soon.
>
> However, I'm good with your allocateTempFileName() changeset,
> so it's up to you to give it a try or not.
>
> > Thanks again,
> > Philip
>
> Thank you,
> Andrey.
Thanks,
Philip
