+1
On Tue, May 3, 2016 at 7:29 PM, Kevin Fenzi <kevin@xxxxxxxxx> wrote:
> Greetings.
>
> There was some talk on the mirror lists about making a fullfilelist
> version that also had timestamps in it so mirrors could look for all
> the files that changed since the last time they synced and only sync
> those specific files. This would be MUCH faster for them as they don't
> need to pull the entire metadata over. This would be MUCH better for us
> as they don't have to hammer our netapps to get all the metadata.
>
> Of course this is not a 100% solution, as they won't see deleted files
> or hardlinks. So, they will likely still need to do a real full sync
> every once in a while (monthly?) to get those things synced up.
>
> I'd like to push the following change to create the file and then we
> can look at writing up some simple scripting for people to use and see
> if we can get people to use it and lower our netapp metadata hammering.
>
> +1s?
>
> kevin
> --
> diff --git a/roles/bodhi2/backend/files/update-fullfilelist b/roles/bodhi2/backend/files/update-fullfilelist
> index 0302c6a..bac3f9c 100755
> --- a/roles/bodhi2/backend/files/update-fullfilelist
> +++ b/roles/bodhi2/backend/files/update-fullfilelist
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -# currently runs on releng2.fedora.phx.redhat.com
> +# currently runs on bodhi-backend01 after updates pushes
>
> MOD=$1
> [ -z "$MOD" ] && {
> @@ -8,6 +8,8 @@ MOD=$1
> exit 1
> }
>
> +# This is the old traditional fullfilelist with no timestamps
> +
> TMPFILE=$(mktemp -p /tmp/)
> pushd /pub/$MOD > /dev/null
> find * -print > $TMPFILE
> @@ -18,3 +20,16 @@ else
> fi
> chmod 0644 fullfilelist
> popd > /dev/null
> +
> +# This is the new list with timestamps
> +
> +TMPFILE=$(mktemp -p /tmp/)
> +pushd /pub/$MOD > /dev/null
> +/usr/bin/rsync --no-h --list-only -r . > $TMPFILE
> +if diff $TMPFILE fullfiletimelist > /dev/null; then
> + rm -f $TMPFILE
> +else
> + mv $TMPFILE fullfiletimelist
> +fi
> +chmod 0644 fullfiletimelist
> +popd > /dev/null
> diff --git a/roles/releng/files/update-fullfilelist b/roles/releng/files/update-fullfilelist
> index 0302c6a..bac3f9c 100755
> --- a/roles/releng/files/update-fullfilelist
> +++ b/roles/releng/files/update-fullfilelist
> @@ -1,6 +1,6 @@
> #!/bin/bash
>
> -# currently runs on releng2.fedora.phx.redhat.com
> +# currently runs on bodhi-backend01 after updates pushes
>
> MOD=$1
> [ -z "$MOD" ] && {
> @@ -8,6 +8,8 @@ MOD=$1
> exit 1
> }
>
> +# This is the old traditional fullfilelist with no timestamps
> +
> TMPFILE=$(mktemp -p /tmp/)
> pushd /pub/$MOD > /dev/null
> find * -print > $TMPFILE
> @@ -18,3 +20,16 @@ else
> fi
> chmod 0644 fullfilelist
> popd > /dev/null
> +
> +# This is the new list with timestamps
> +
> +TMPFILE=$(mktemp -p /tmp/)
> +pushd /pub/$MOD > /dev/null
> +/usr/bin/rsync --no-h --list-only -r . > $TMPFILE
> +if diff $TMPFILE fullfiletimelist > /dev/null; then
> + rm -f $TMPFILE
> +else
> + mv $TMPFILE fullfiletimelist
> +fi
> +chmod 0644 fullfiletimelist
> +popd > /dev/null
>
> _______________________________________________
> infrastructure mailing list
> infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
> http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
>
_______________________________________________
infrastructure mailing list
infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
