Greetings.
There was some talk on the mirror lists about making a fullfilelist
version that also had timestamps in it so mirrors could look for all
the files that changed since the last time they synced and only sync
those specific files. This would be MUCH faster for them as they don't
need to pull the entire metadata over. This would be MUCH better for us
as they don't have to hammer our netapps to get all the metadata.
Of course this is not a 100% solution, as they won't see deleted files
or hardlinks. So, they will likely still need to do a real full sync
every once in a while (monthly?) to get those things synced up.
I'd like to push the following change to create the file and then we
can look at writing up some simple scripting for people to use and see
if we can get people to use it and lower our netapp metadata hammering.
+1s?
kevin
--
diff --git a/roles/bodhi2/backend/files/update-fullfilelist b/roles/bodhi2/backend/files/update-fullfilelist
index 0302c6a..bac3f9c 100755
--- a/roles/bodhi2/backend/files/update-fullfilelist
+++ b/roles/bodhi2/backend/files/update-fullfilelist
@@ -1,6 +1,6 @@
#!/bin/bash
-# currently runs on releng2.fedora.phx.redhat.com
+# currently runs on bodhi-backend01 after updates pushes
MOD=$1
[ -z "$MOD" ] && {
@@ -8,6 +8,8 @@ MOD=$1
exit 1
}
+# This is the old traditional fullfilelist with no timestamps
+
TMPFILE=$(mktemp -p /tmp/)
pushd /pub/$MOD > /dev/null
find * -print > $TMPFILE
@@ -18,3 +20,16 @@ else
fi
chmod 0644 fullfilelist
popd > /dev/null
+
+# This is the new list with timestamps
+
+TMPFILE=$(mktemp -p /tmp/)
+pushd /pub/$MOD > /dev/null
+/usr/bin/rsync --no-h --list-only -r . > $TMPFILE
+if diff $TMPFILE fullfiletimelist > /dev/null; then
+ rm -f $TMPFILE
+else
+ mv $TMPFILE fullfiletimelist
+fi
+chmod 0644 fullfiletimelist
+popd > /dev/null
diff --git a/roles/releng/files/update-fullfilelist b/roles/releng/files/update-fullfilelist
index 0302c6a..bac3f9c 100755
--- a/roles/releng/files/update-fullfilelist
+++ b/roles/releng/files/update-fullfilelist
@@ -1,6 +1,6 @@
#!/bin/bash
-# currently runs on releng2.fedora.phx.redhat.com
+# currently runs on bodhi-backend01 after updates pushes
MOD=$1
[ -z "$MOD" ] && {
@@ -8,6 +8,8 @@ MOD=$1
exit 1
}
+# This is the old traditional fullfilelist with no timestamps
+
TMPFILE=$(mktemp -p /tmp/)
pushd /pub/$MOD > /dev/null
find * -print > $TMPFILE
@@ -18,3 +20,16 @@ else
fi
chmod 0644 fullfilelist
popd > /dev/null
+
+# This is the new list with timestamps
+
+TMPFILE=$(mktemp -p /tmp/)
+pushd /pub/$MOD > /dev/null
+/usr/bin/rsync --no-h --list-only -r . > $TMPFILE
+if diff $TMPFILE fullfiletimelist > /dev/null; then
+ rm -f $TMPFILE
+else
+ mv $TMPFILE fullfiletimelist
+fi
+chmod 0644 fullfiletimelist
+popd > /dev/null
Attachment:
pgpf1Z3FUY3y2.pgp
Description: OpenPGP digital signature
_______________________________________________ infrastructure mailing list infrastructure@xxxxxxxxxxxxxxxxxxxxxxx http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
