Table of Contents
This is an old revision of the document!
Instruction for backup
Created by Alejandro Cabrales
Brief description of backup procedure
Each person is responsible to create a git repository under gitlab, and perform periodic backups that will be pushed towards zapad or an equivalent server. When you login into gitlab in an internet browser you will be able to see the history of your backups. Git is not easy or intuitive, but fortunately you do not need to master it to use it for common backups.
1. Creating a Git repository
- You first choose the folder where you will create your repository. If you are afraid and need some practice, you can start with an empty folder. Get into that folder and type
- This command will create a hidden file named .git, which will contain the different commits that you make (do not worry if you do not understand what a commit is. Hopefully it will be clear as you read more… and practice!).
2. Add files that you want to include into the backup
- Now it is time to add the files that you want to track for backups. Here you can add both files and directories. Do not add anything that you do not want in the backup.
- For the practice, create two text files in your empty directory, say myfile1.txt and myfile2.txt. Then add the former using
git add myfile1.txt
- You can type “git status” to verify that myfile1.txt was added and ready to be committed, whereas myfile2.txt remains untracked. Here myfile1.txt represents files and folders suitable for being backup, such as source codes, makefiles and possibly parameters files. On the contrary, myfile2.txt represents all that you should not incorporate into the backup, such as binaries, results, etc.
3. Commit your changes
- Once you added the files, you create the local backup using git commit:
git commit -m "This is my backup"
- Notice that I have added label “-m” which allows to write a message or comment associated to the backup. I strongly recommend you to take advantage of this, for it is easy to lose track of the changes that one makes in one's codes. If you are constantly changing you code, either debugging or testing, it can be wise to commit, for instance, before any “major surgery” that you are palling in your codes, commenting the current status and what you are about to do.
4. Push to remote repository
- The final step is to push the local repository to the server.
Introduction of rsync and crontab
- Introduction of rsync
rsync is a software application and network protocol for Unix-like and Windows systems that synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. In the backup procedure, rsync is used to synchronize your files and directories from the server to your local computer.
- Introduction of crontab
A cron is a utility that allows tasks to automatically run in the background of the system at regular intervals by use of the cron daemon. Crontab (CRON TABle) is a file, which contains the schedule of cron entries to be run, and at what times they are to be run. In the backup procedure, crontab is quite useful, which helps you do the backup once a day without manual operation.
Detailed description of the workflow
- 1. Make a directory in your local computer for backup
e.g. mkdir ~/backup
- 2. Copy the rsync script to your backup directory
Download the python script backup.py (click on the name).
- 3. Change the script
There are two major parts that need to be changed in the script backup.py (currently the web backup is not performed, thus I commented the corresponding lines out). Such parts are circled in the following figure.
The upper red circle means the directory in your local mac that you want to back up to. The lower red circle means the directory in the server that you want to back up. In this example I back up my home directory in oas, cabrales. You can change them into your desired directory. Don’t forget to remove/add the for loop in the script if you plan to back up other than this directory, and change the server name accordingly. I use oas in this example (circled in blue). Notice if your user name on your mac is different from the user name on oas, you need to add the oas user name in the blue circle, i.e. from oas ⇒ my_oas_username@oas.
- 4. [Optional] Set up a notification
Disregard this. Growl is no longer freeware.
- 5. Create a cron schedule
- (1) Type “crontab –e” in the terminal window.
The text editor will open a blank window for the “crontab entries” to be entered. Each line presents a separate cron jobs.
- (2) Put the following line in the window and remember to change the directory
* * * * * /Users/YourNameInMacDirectory/backup/backup.py
If you used vi for the edition, when you exit you'll get the message: “crontab: installing new crontab.”
An asterisk (*) is used to indicate that every instance (i.e. every hour, every weekday, etc.) of the particular time period will be used. Here is how positions 1-5 are layed out:
|2||Hour 0-23 (0 = midnight)|
|5||Weekday 0-6 (0 = Sunday)|
- (3) Change asterisks (*) accordingly
Find you name and the time for you to back up in the following table. The starting time means your backup script will automatically run at this particular time. Then change * * * * * according to the corresponding format. For example, my command would look:
40 3 * * * /Users/alejandrocabrales/backup/backup.py
|Fantine Huot||00:00 am||0 0 * * *|
|Joe Jennings||00:20 am||20 0 * * *|
|Taylor Dahlke||00:40 am||40 0 * * *|
|Guillaume Barnier||01:00 am||0 1 * * *|
|Ettore Biondi||01:20 am||20 1 * * *|
|Yinbin Ma||01:40 am||40 1 * * *|
|Jon Claerbout||02:00 am||0 2 * * *|
|Biondo Biondi||02:20 am||20 2 * * *|
|Rahul Sarkar||02:40 am||40 2 * * *|
|Bob Clapp||03:00 am||0 3 * * *|
|Jason Chang||03:20 am||20 3 * * *|
|Alejandro Cabrales||03:40 am||40 3 * * *|
|Huy Le||04:00 am||0 4 * * *|
|Stuart Farris||04:20 am||20 4 * * *|
|Eileen Martin||04:40 am||40 4 * * *|
|Rustam Akhmadiev||05:00 am||0 5 * * *|
|Siyuan Yuan||05:20 am||20 5 * * *|
|Stew Levin||05:40 am||40 5 * * *|
|Shuki Ronen||06:00 am||0 6 * * *|
|Miguel Ferrier||06:20 am||20 6 * * *|
IMPORTANT: Crontab won't work unless your computer is awake.
- 6. [Optional] Check the success of your crontab
Type “ls -ltr /var/mail”, and check the time in the following information
"-rw------- 1 yishen mail 5559 Mar 14 1:40 yishen"
The first backup may take a while, possibly exceeding 20 minutes. Manually running the script for your first backup during the weekend or in the evening instead of using crontab is recommended. To run the script manually, type the following command in your back up directory.
(Ps: probably you will be asked to connect to the server at your first time.)