Umm.... uh oh!
In testing out the process for replicating/copying a base server to a
smaller target server (droplet/vm) for Digital Ocean (DO), I've hit a
major snag.
It appears that copying "some" dirs/files screws up the target vm....
So, I'm trying to work my way through to see just what can be copied
without screwing up the target vm/droplet.
The files/dirs that I exclude are in the "skiprsync.dat" file. At the
moment, I'm doing an iterative process to see how much/what the skip
file should contain.
The process for now...
1- create target droplet/vm
2- modify the skipdirs.rsync file with the files/dirs to skip during the rsync
3- run the rsync to xfer files from the base to the target droplet..
4- reboot the target droplet to see if it can still be accessed/ssh
5- if not, repeat step 1...
thoughts/comments are appreciated.
skipdirs.rsync::
/proc/*
/sys/*
/dev/*
/media/*
/var/log/*
/var/log/journal/*
**/.cache/google-chrome/***
**/.ccache/***
/BACKUPS/*
/run/media/*
/var/lib/nfs/*
/usr/src/kernels/*
/root/.cache/*
/swapfile
/bin/*
/root
/home/*
/home/crawl_user
/home/root1
/boot/*
/mnt/*
/var/*
/usr/*
/run/*
/etc/*
/etc/fstab
/etc/exports
/etc/mtab
/etc/passwd
/etc/group
/etc/dbus-1/*
/etc/grub.d/*
/etc/iproute2/*
/etc/kernel/*
/etc/selinux/*
/etc/sysconfig/*
/etc/systemd/*
/etc/tuned/*
/etc/udev/*
/etc/wpa_suplicant/*
/etc/xdg/*
On Sun, Feb 11, 2018 at 4:35 PM, InvalidPath <invalid.path@xxxxxxxxx> wrote:
>
>
> On Sun, Feb 11, 2018 at 1:51 PM, bruce <badouglas@xxxxxxxxx> wrote:
>>
>> Hi.
>>
>> Kind of long ,but might be useful/helpful to others.. Feel free to
>> comment as you see fit!
>>
>> I'm using this as a step/guide to recreate/replicate a smaller
>> droplet/vm from a larger vm within Digital Ocean.
>>
>>
>> The goal:
>> Replicate/reproduce the users/processes/directories/files
>> from a base server to a target server.
>> To be able to then use the target server in place of the base/initial
>> server
>> The base server acts as a nfsClient server to a nfsServer
>>
>> Doing this to create a smaller vm/droplet from Digital Ocean (DO), and
>> need
>> to "replicate" a larger droplet/vm.
>> DO doesn't provide a way to accomplish this!
>>
>> ::::---
>> This process doesn't deal/handle any iptable/firewall/selinux/dns
>> issues....
>> -need to figure out how to deal with these....
>>
>> Process:
>> -analyse the base server to get all the existing users
>> -analyse the base server to get all the existing/running system processes
>> -analyse the base server to get all the existing/running 3rd party
>> processes
>> -analyse the base server to get all the existing dirs/files
>> -analyse the base server to get all the existing sshkey data/files
>> -analyse the base server to get all the installed rpm/packages
>>
>>
>> -create process on the target to generate the user/group/passwd
>> for the users on the base server
>> -create process on the target to generate the ssh key for the users
>> to replicate the base users/sshkeys
>> -create process to copy all the dir/files from the base to the target,
>> excluding a limited subset of dirs..
>> -create process to install on the target, all the installed packages from
>> the base
>> -
>>
>>
>> Steps::
>> -on the target disable selinux for simplicity
>> vi /etc/sysconfig/selinux
>> set >> SELINUX=disabled <<
>>
>>
>> -Create the list of users
>> -On the target, generate the users/passwd/groups. It appears this
>> could/should be doable by copying the requisite files from the
>> base->target with the associated perms..
>> Decided to do it manually to ensure it matches..
>>
>> -the users are/were:
>> root/root_tmp/test_user
>>
>> -use useradd/passwd to generate the same users/passwd/group as well as
>> the same uid/gid for the userID/groupID to match the base server
>>
>> --match the user/passwd/uid/gid so the target matches the base
>> --if required, mod the uid/gid
>> usermod -u xx test_user
>> groupmod -g xx test_user
>>
>> --at the same time, change/match across the entire dir for any files
>> to handle user/group owner (do this for all users)
>> find / -group 500 -exec chgrp -h test_user {} \;
>> find / -user 500 -exec chown -h test_user {} \;
>>
>> -change to given user (root/root_tmp/test_user)
>> set up the ssh keymkdir ~/.ssh
>> chmod 700 ~/.ssh
>> echo '' > ~/.ssh/authorized_keys
>> chmod 600 ~/.ssh/authorized_keys
>>
>> chmod 755 ~ ~/.ssh
>>
>> update/cpy in the "authorized_keys" file the pub key from the
>> base/user "authorized_keys" file
>>
>> at this point, the users are set, and the ssh keys are set..
>>
>> ----------------------------------
>> Copy the dirs/files....
>>
>> The process runs on the base, copying/rsync from the base-> target
>> handling
>> the entire disk from the "/" top.. on down..
>>
>> The excluded list follows as well as as the rsync cmd..
>>
>> The file "/etc/skipdirs.rsync" contains:
>>
>> /proc/*
>> /sys/*
>> /dev/*
>> /media/*
>> /var/log/*
>> /var/log/journal/*
>> **/.cache/google-chrome/***
>> **/.ccache/***
>> /BACKUPS/*
>> /run/media/*
>> /var/lib/nfs/*
>> /usr/src/kernels/*
>> /root/.cache/*
>> /swapfile
>> /bin/*
>>
>> rsync --progress -avAI --exclude-from=/etc/skipdirs.rsync /
>> root@11.22.33.44:/
>> (shout out to Rick!!)
>>
>> I intentionally wanted to see all the files xfered as I ran/run the cmd..
>> -the cmd is run as "root" on the base, to ensure the process has
>> complete access to all dirs/files..
>> -the "root" user on the target has access to top level as well..
>>
>> RPM packages..
>> -To further ensure the target will mtach the base,
>> on the base as "root" run
>> rpm -qa > rpmlist.dat
>> rsync --progress -avAI /rpmlist.dat root@11.22.33.44:/
>>
>> and on the target as root
>> cat /rpmlist.dat | xargs yum -y install
>>
>> followed by
>> yum update
>>
>> --end result is that all the packages on the target should match
>> the base
>>
>> ::given that this is going to be a "client" to the nfsServer..
>> -make sure the nfs client utils are available.. they should already
>> be but check anyway..
>>
>> on the target as root
>>
>> install any/all additional packages on the target::
>> yum install nfs-utils nfs-utils-lib
>>
>> as root
>> sudo cat << EOF > /etc/yum.repos.d/google-chrome.repo
>> [google-chrome]
>> name=google-chrome - \$basearch
>> baseurl=http://dl.google.com/linux/chrome/rpm/stable/\$basearch
>> enabled=1
>> gpgcheck=1
>> gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
>> EOF
>>
>> yum install -y gcc xorg-x11-server-Xvfb google-chrome-stable
>> yum install xvfb x11-xkb-utils
>> yum install xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic
>>
>> yum install vi
>> pip install pyvirtualdisplay
>> python -m pip install --upgrade pip setuptools wheel
>>
>> easy_install simplejson
>> yum install libxml2-python
>>
>> yum install epel-release
>> pip install -U selenium
>> pip install pyvirtualdisplay
>> yum install Xvfb libXfont Xorg
>>
>>
>> -finally.. install the "stuff" for the selenium/firefox/chrome testing..
>>
>> wget
>> https://github.com/mozilla/geckodriver/releases/download/v0.19.1/geckodriver-v0.19.1-linux64.tar.gz
>> tar -x geckodriver -zf geckodriver-v0.19.1-linux64.tar.gz -O >
>> /usr/local/bin/geckodriver
>> chmod +x /usr/local/bin/geckodriver
>> rm geckodriver-v0.19.1-linux64.tar.gz
>> ln -s /path/to/file /path/to/symlink
>> ln -s /usr/local/bin/geckodriver /usr/bin/geckodriver
>>
>> ## Chromedriver
>> wget
>> https://chromedriver.storage.googleapis.com/2.35/chromedriver_linux64.zip
>> unzip chromedriver_linux64.zip
>> sudo chmod +x chromedriver
>> sudo mv chromedriver /usr/local/bin/
>> rm chromedriver_linux64.zip
>> ln -s /usr/local/bin/chromedriver /usr/bin/chromedriver
>>
>>
>>
>> finally.... on the base....
>> as root..
>> vi /etc/ssh/sshd_config
>> #PermitRootLogin yes
>> PermitRootLogin no
>>
>>
>> ok....
>>
>>
>> if you can see something else that can be done that I might have
>> missed.. or that would be better.. shout it out.
>>
>> At some point.. others on DO will be looking for something like this.
>>
>> Ideally, it would be cool/good to have this in a kind of chef/puppet
>> cookbook.
>>
>> But that's above my pay grade for now!
>>
>> thanks
>> _______________________________________________
>> users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
>> To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
>
>
> I am keeping this for posterity.. thanks for such a well rounded
> step-by-step.
>
> _______________________________________________
> users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
>
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
