Re: cephfs snapshot scripting questions

["Marc Roos" <M.Roos@xxxxxxxxxxxxxxxxx>]

Hmmmm, ok ok, test it first, can't remember if it is finished. Checks 
also if it is usefull to create a snapshot, by checking the size of the 
directory.

[@ cron.daily]# cat backup-archive-mail.sh
#!/bin/bash


cd /home/

for account in `ls -c1 /home/mail-archive/ | sort`
do
  /usr/local/sbin/backup-snap.sh mail-archive/$account 7
  /usr/local/sbin/backup-snap.sh archiveindex/$account 7
done 



[@ cron.daily]# cat /usr/local/sbin/backup-snap.sh
#!/bin/bash
#
# usage: ./backup-snap.sh dirname ret
#
# dont forget to change
# START_HOURS_RANGE=0-22
#
# command to backup:


# static variables
# BACKUPDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
BACKUPDIR="$PWD"
BACKUPPREFIX="snap"
CEPHSDIR=".snap"
ARCHIVEDAYS=30
DOW=`date +%u`
DOM=`date +%-d`

if [ $# -lt 2 ]
then
 echo ""
 echo "usage: $0 dirname ret-days "
 echo ""
 echo ""
exit;
fi

# used for hardlinking with rsync to previous version
PREVDAY=
# current version backup location
VB=
# argument variables
ARGS=$*
SNAPBAK=$1
BACKUPDAYS=$2
ARCHIVE=0

function setpreviousday {

PREVDAY=`expr $DOM - 1`

if [ $PREVDAY -eq 0 ]
then
  PREVDAY=$BACKUPDAYS
fi
}

function getdirsize {
local path="$1"
local size=0
local rval=0

size=$(getfattr --only-values --absolute-names -d -m ceph.dir.rbytes 
"$path" 2> /dev/null)
if [ $? -eq 0 ]
then
  rval=$size
fi

echo -n $rval
}

function resetdom {
# reset the dom counter so XX days are backuped

if [ $DOM -gt $BACKUPDAYS ]
then
  DOM=`expr $DOM - $BACKUPDAYS`
  resetdom
fi

}

function removewhitespace {
NEW=`echo -n "$1" | sed -e 's/\s//g' `
echo -n "$NEW"
}


function createsnapshot () {
  SNAPBAK="$1"
  SNNAME="$2"

  mkdir "$BACKUPDIR/$SNAPBAK/$CEPHSDIR/$SNNAME"
}

function snapshotremove () {
  SNAPBAK="$1"
  SNNAME="$2"

  #echo "removing snapshot $SNAPBAK"
  rmdir "$BACKUPDIR/$SNAPBAK/$CEPHSDIR/$SNNAME"
  if [ $? -ne 0 ]
  then
    echo "error removing old snapshot $SNAPBAK"
    exit;
  fi
  sleep 2
}

function snapshotexists () {
# returns 0 if exists
  rval=1
  SNAPBAK="$1"
  SNNAME="$2"

  FOUND="$BACKUPDIR/$SNAPBAK/$CEPHSDIR/$SNNAME"
  if [ -d $FOUND ]
  then
    rval=0
  fi

  echo -n $rval
}


# script arguments
#setargs

umask 0027


# reset day of month in scale of 1 to BACKUPDAYS
resetdom
VB=$DOM

# calculate previous day number
setpreviousday

# do server backups
SNNAME=$BACKUPPREFIX"-"$VB


# do only snapshot if there is data
if [ $(getdirsize "$SNAPBAK") -ne 0 ]
then
  if [ $(snapshotexists $SNAPBAK $SNNAME) -eq 0 ]
  then
    snapshotremove $SNAPBAK $SNNAME
  fi

  createsnapshot $SNAPBAK $SNNAME
fi





-----Original Message-----
From: Robert Ruge [mailto:robert.ruge@xxxxxxxxxxxxx] 
Sent: woensdag 17 juli 2019 2:44
To: ceph-users@xxxxxxxxxxxxxx
Subject:  cephfs snapshot scripting questions

Greetings.

 

Before I reinvent the wheel has anyone written a script to maintain X 
number of snapshots on a cephfs file system that can be run through 
cron?

I am aware of the cephfs-snap code but just wondering if there are any 
other options out there.

 

On a related note which of these options would be better?

1.       Maintain one .snap directory at the root of the cephfs tree - 
/ceph/.snap

2.       Have a .snap directory for every second level directory 
/ceph/user/.snap

 

I am thinking the later might make it more obvious for the users to do 
their own restores but wondering what the resource implications of 
either approach might be.

 

The documentation indicates that I should use kernel >= 4.17 for cephfs. 
 I’m currently using Mimic 13.2.6 on Ubuntu 18.04 with kernel version 
4.15.0. What issues might I see with this combination? I’m hesitant to 
upgrade to an unsupported kernel on Ubuntu but wondering if I’m going 
to be playing Russian Roulette with this combo. 

 

Are there any gotcha’s I should be aware of before plunging into full 
blown cephfs snapshotting?

 

Regards and thanks.

Robert Ruge

 


Important Notice: The contents of this email are intended solely for the 
named addressee and are confidential; any unauthorised use, reproduction 
or storage of the contents is expressly prohibited. If you have received 
this email in error, please delete it and any attachments immediately 
and advise the sender by return email or telephone.

Deakin University does not warrant that this email and any attachments 
are error or virus free. 


_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com