On 6/8/07, Han-Wen Nienhuys <hanwen@xxxxxxxxx> wrote:
Benjamin Sergeant escreveu:
> So are you saying that in the old days, git-p4 was importing the p4
> depo in small slices to not overkill the process memory (in case the
> depo is big) ?
no, in the "old days" git-p4 used a separate p4 invocation for each file.
Anyway, in case you hit command line lenght limit here it is. That
might be interesting for the "next days" :)
Benjamin.
[bsergean@flanders fast-export]$ git format-patch -k -m --stdout origin
From 45f2dbdb9a8c0b3beb007ae892613cdc4afab80a Mon Sep 17 00:00:00 2001
From: Benjamin Sergeant <bsergean@flanders.(none)>
Date: Fri, 8 Jun 2007 09:58:57 -0700
Subject: Split p4 print call into multiple call to not exceed the
command line lenght maximum
---
git-p4 | 21 ++++++++++++++++++---
1 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/git-p4 b/git-p4
index 36fe69a..906b193 100755
--- a/git-p4
+++ b/git-p4
@@ -703,9 +703,22 @@ class P4Sync(Command):
if not files:
return
- filedata = p4CmdList('print %s' % ' '.join(['"%s#%s"' % (f['path'],
- f['rev'])
- for f in files]))
+ # We cannot put all the files on the command line
+ # OS have limitations on the max lenght of arguments
+ # POSIX says it's 4096 bytes, default for Linux seems to be 130 K.
+ # and all OS from the table below seems to be higher than POSIX.
+ # See http://www.in-ulm.de/~mascheck/various/argmax/
+ chunk = ''
+ filedata = []
+ for i in xrange(len(files)):
+ f = files[i]
+ chunk += '"%s#%s" ' % (f['path'], f['rev'])
+ if len(chunk) > 4000 or i == len(files)-1:
+ data = p4CmdList('print %s' % chunk)
+ if "p4ExitCode" in data[0]:
+ die("Problems executing p4. Error: [%d]." %
(data[0]['p4ExitCode']));
+ filedata.extend(data)
+ chunk = ''
j = 0;
contents = {}
@@ -1486,3 +1499,5 @@ def main():
if __name__ == '__main__':
main()
+
+# vim: set filetype=python sts=4 sw=4 et si :
--
1.5.0.4
From dd9975708433efeec37b608755f54fbeaedf0f3f Mon Sep 17 00:00:00 2001
From: Benjamin Sergeant <bsergean@flanders.(none)>
Date: Fri, 8 Jun 2007 10:20:39 -0700
Subject: Use os.sysconf('SC_ARG_MAX') to retrieve the max value, and
build the string using join (faster)
---
git-p4 | 17 ++++++++++-------
1 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/git-p4 b/git-p4
index 906b193..8dc1963 100755
--- a/git-p4
+++ b/git-p4
@@ -705,20 +705,23 @@ class P4Sync(Command):
# We cannot put all the files on the command line
# OS have limitations on the max lenght of arguments
- # POSIX says it's 4096 bytes, default for Linux seems to be 130 K.
- # and all OS from the table below seems to be higher than POSIX.
# See http://www.in-ulm.de/~mascheck/various/argmax/
- chunk = ''
+ chunks = []
+ chunkLenght = 0
filedata = []
+ maxlenght = max(int(os.sysconf('SC_ARG_MAX') * 0.90), 4000)
+ print maxlenght
for i in xrange(len(files)):
f = files[i]
- chunk += '"%s#%s" ' % (f['path'], f['rev'])
- if len(chunk) > 4000 or i == len(files)-1:
- data = p4CmdList('print %s' % chunk)
+ chunkLenght += len(f['path']) + len(f['rev'])
+ chunks.append('"%s#%s" ' % (f['path'], f['rev']))
+ if chunkLenght > maxlenght or i == len(files)-1:
+ data = p4CmdList('print %s' % ' '.join(chunks))
if "p4ExitCode" in data[0]:
die("Problems executing p4. Error: [%d]." %
(data[0]['p4ExitCode']));
filedata.extend(data)
- chunk = ''
+ chunks = []
+ chunkLenght = 0
j = 0;
contents = {}
--
1.5.0.4
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
