On Wed, May 11, 2016 at 09:21:27AM -0500, Dennis Gilmore wrote:
> On Tuesday, May 3, 2016 1:29:50 PM CDT Kevin Fenzi wrote:
> > Greetings.
> >
> > There was some talk on the mirror lists about making a fullfilelist
> > version that also had timestamps in it so mirrors could look for all
> > the files that changed since the last time they synced and only sync
> > those specific files. This would be MUCH faster for them as they don't
> > need to pull the entire metadata over. This would be MUCH better for us
> > as they don't have to hammer our netapps to get all the metadata.
> >
> > Of course this is not a 100% solution, as they won't see deleted files
> > or hardlinks. So, they will likely still need to do a real full sync
> > every once in a while (monthly?) to get those things synced up.
> >
> > I'd like to push the following change to create the file and then we
> > can look at writing up some simple scripting for people to use and see
> > if we can get people to use it and lower our netapp metadata hammering.
> >
> > +1s?
> >
> > kevin
> > --
> > diff --git a/roles/bodhi2/backend/files/update-fullfilelist
> > b/roles/bodhi2/backend/files/update-fullfilelist index 0302c6a..bac3f9c
> > 100755
> > --- a/roles/bodhi2/backend/files/update-fullfilelist
> > +++ b/roles/bodhi2/backend/files/update-fullfilelist
> > @@ -1,6 +1,6 @@
> > #!/bin/bash
> >
> > -# currently runs on releng2.fedora.phx.redhat.com
> > +# currently runs on bodhi-backend01 after updates pushes
> >
> > MOD=$1
> > [ -z "$MOD" ] && {
> > @@ -8,6 +8,8 @@ MOD=$1
> > exit 1
> > }
> >
> > +# This is the old traditional fullfilelist with no timestamps
> > +
> > TMPFILE=$(mktemp -p /tmp/)
> > pushd /pub/$MOD > /dev/null
> > find * -print > $TMPFILE
> > @@ -18,3 +20,16 @@ else
> > fi
> > chmod 0644 fullfilelist
> > popd > /dev/null
> > +
> > +# This is the new list with timestamps
> > +
> > +TMPFILE=$(mktemp -p /tmp/)
> > +pushd /pub/$MOD > /dev/null
> > +/usr/bin/rsync --no-h --list-only -r . > $TMPFILE
> > +if diff $TMPFILE fullfiletimelist > /dev/null; then
> > + rm -f $TMPFILE
> > +else
> > + mv $TMPFILE fullfiletimelist
> > +fi
> > +chmod 0644 fullfiletimelist
> > +popd > /dev/null
> > diff --git a/roles/releng/files/update-fullfilelist
> > b/roles/releng/files/update-fullfilelist index 0302c6a..bac3f9c 100755
> > --- a/roles/releng/files/update-fullfilelist
> > +++ b/roles/releng/files/update-fullfilelist
> > @@ -1,6 +1,6 @@
> > #!/bin/bash
> >
> > -# currently runs on releng2.fedora.phx.redhat.com
> > +# currently runs on bodhi-backend01 after updates pushes
> >
> > MOD=$1
> > [ -z "$MOD" ] && {
> > @@ -8,6 +8,8 @@ MOD=$1
> > exit 1
> > }
> >
> > +# This is the old traditional fullfilelist with no timestamps
> > +
> > TMPFILE=$(mktemp -p /tmp/)
> > pushd /pub/$MOD > /dev/null
> > find * -print > $TMPFILE
> > @@ -18,3 +20,16 @@ else
> > fi
> > chmod 0644 fullfilelist
> > popd > /dev/null
> > +
> > +# This is the new list with timestamps
> > +
> > +TMPFILE=$(mktemp -p /tmp/)
> > +pushd /pub/$MOD > /dev/null
> > +/usr/bin/rsync --no-h --list-only -r . > $TMPFILE
> > +if diff $TMPFILE fullfiletimelist > /dev/null; then
> > + rm -f $TMPFILE
> > +else
> > + mv $TMPFILE fullfiletimelist
> > +fi
> > +chmod 0644 fullfiletimelist
> > +popd > /dev/null
> -1
>
> It is not a good solution.
Could you maybe explain a little more what this means?
How is it not a good solution? What would be a better one?
After all this is coming as a solution to a problem raised by one or two mirror
admins, it sounds reasonable to me and I'm not seeing how it would be a bad
solution.
Pierre
_______________________________________________
infrastructure mailing list
infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
