+1.
On 4 May 2016 at 11:44, Patrick Uiterwijk <puiterwijk@xxxxxxxxxx> wrote:
> +1
>
> On Tue, May 3, 2016 at 7:29 PM, Kevin Fenzi <kevin@xxxxxxxxx> wrote:
>> Greetings.
>>
>> There was some talk on the mirror lists about making a fullfilelist
>> version that also had timestamps in it so mirrors could look for all
>> the files that changed since the last time they synced and only sync
>> those specific files. This would be MUCH faster for them as they don't
>> need to pull the entire metadata over. This would be MUCH better for us
>> as they don't have to hammer our netapps to get all the metadata.
>>
>> Of course this is not a 100% solution, as they won't see deleted files
>> or hardlinks. So, they will likely still need to do a real full sync
>> every once in a while (monthly?) to get those things synced up.
>>
>> I'd like to push the following change to create the file and then we
>> can look at writing up some simple scripting for people to use and see
>> if we can get people to use it and lower our netapp metadata hammering.
>>
>> +1s?
>>
>> kevin
>> --
>> diff --git a/roles/bodhi2/backend/files/update-fullfilelist b/roles/bodhi2/backend/files/update-fullfilelist
>> index 0302c6a..bac3f9c 100755
>> --- a/roles/bodhi2/backend/files/update-fullfilelist
>> +++ b/roles/bodhi2/backend/files/update-fullfilelist
>> @@ -1,6 +1,6 @@
>> #!/bin/bash
>>
>> -# currently runs on releng2.fedora.phx.redhat.com
>> +# currently runs on bodhi-backend01 after updates pushes
>>
>> MOD=$1
>> [ -z "$MOD" ] && {
>> @@ -8,6 +8,8 @@ MOD=$1
>> exit 1
>> }
>>
>> +# This is the old traditional fullfilelist with no timestamps
>> +
>> TMPFILE=$(mktemp -p /tmp/)
>> pushd /pub/$MOD > /dev/null
>> find * -print > $TMPFILE
>> @@ -18,3 +20,16 @@ else
>> fi
>> chmod 0644 fullfilelist
>> popd > /dev/null
>> +
>> +# This is the new list with timestamps
>> +
>> +TMPFILE=$(mktemp -p /tmp/)
>> +pushd /pub/$MOD > /dev/null
>> +/usr/bin/rsync --no-h --list-only -r . > $TMPFILE
>> +if diff $TMPFILE fullfiletimelist > /dev/null; then
>> + rm -f $TMPFILE
>> +else
>> + mv $TMPFILE fullfiletimelist
>> +fi
>> +chmod 0644 fullfiletimelist
>> +popd > /dev/null
>> diff --git a/roles/releng/files/update-fullfilelist b/roles/releng/files/update-fullfilelist
>> index 0302c6a..bac3f9c 100755
>> --- a/roles/releng/files/update-fullfilelist
>> +++ b/roles/releng/files/update-fullfilelist
>> @@ -1,6 +1,6 @@
>> #!/bin/bash
>>
>> -# currently runs on releng2.fedora.phx.redhat.com
>> +# currently runs on bodhi-backend01 after updates pushes
>>
>> MOD=$1
>> [ -z "$MOD" ] && {
>> @@ -8,6 +8,8 @@ MOD=$1
>> exit 1
>> }
>>
>> +# This is the old traditional fullfilelist with no timestamps
>> +
>> TMPFILE=$(mktemp -p /tmp/)
>> pushd /pub/$MOD > /dev/null
>> find * -print > $TMPFILE
>> @@ -18,3 +20,16 @@ else
>> fi
>> chmod 0644 fullfilelist
>> popd > /dev/null
>> +
>> +# This is the new list with timestamps
>> +
>> +TMPFILE=$(mktemp -p /tmp/)
>> +pushd /pub/$MOD > /dev/null
>> +/usr/bin/rsync --no-h --list-only -r . > $TMPFILE
>> +if diff $TMPFILE fullfiletimelist > /dev/null; then
>> + rm -f $TMPFILE
>> +else
>> + mv $TMPFILE fullfiletimelist
>> +fi
>> +chmod 0644 fullfiletimelist
>> +popd > /dev/null
>>
>> _______________________________________________
>> infrastructure mailing list
>> infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
>> http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
>>
> _______________________________________________
> infrastructure mailing list
> infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
> http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
--
Stephen J Smoogen.
_______________________________________________
infrastructure mailing list
infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
http://lists.fedoraproject.org/admin/lists/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
