Table of Contents
This is an old revision of the document!
Instruction for backup
Created by Alejandro Cabrales
Brief description of backup procedure
Each person is responsible to create a git repository under gitlab, and perform periodic backups that will be pushed towards zapad or an equivalent server. When you login into gitlab in an internet browser you will be able to see the history of your backups. Git is not easy or intuitive, but fortunately you do not need to master it to use it for common backups.
1. Creating a Git repository
- You first choose the folder where you will create your repository. If you are afraid and need some practice, you can start with an empty folder. Get into that folder and type
- This command will create a hidden file named .git, which will contain the different commits that you make (do not worry if you do not understand what a commit is. Hopefully it will be clear as you read more… and practice!).
2. Add files that you want to include into the backup
- Now it is time to add the files that you want to track for backups. Here you can add both files and directories. Do not add anything that you do not want in the backup.
- For the practice, create two text files in your empty directory, say myfile1.txt and myfile2.txt. Then add the former using
git add myfile1.txt
- You can type “git status” to verify that myfile1.txt was added and ready to be committed, whereas myfile2.txt remains untracked. Here myfile1.txt represents files and folders suitable for being backup, such as source codes, makefiles and possibly parameters files. On the contrary, myfile2.txt represents all that you should not incorporate into the backup, such as binaries, results, etc.
3. Commit your changes
- Once you added the files, you create the local backup using git commit:
git commit -m "This is my backup"
- Notice that I have added label “-m” which allows to write a message or comment associated to the backup. I strongly recommend you to take advantage of this, for it is easy to lose track of the changes that one makes in one's codes. If you are constantly changing you code, either debugging or testing, it can be wise to commit, for instance, before any “major surgery” that you are palling in your codes, commenting the current status and what you are about to do.
4. Push to remote repository
- The final step is to push the local repository to the server.
Introduction of rsync and crontab
- Introduction of rsync
rsync is a software application and network protocol for Unix-like and Windows systems that synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. In the backup procedure, rsync is used to synchronize your files and directories from the server to your local computer.
- Introduction of crontab
A cron is a utility that allows tasks to automatically run in the background of the system at regular intervals by use of the cron daemon. Crontab (CRON TABle) is a file, which contains the schedule of cron entries to be run, and at what times they are to be run. In the backup procedure, crontab is quite useful, which helps you do the backup once a day without manual operation.
Detailed description of the workflow
- 1. Make a directory in your local computer for backup
e.g. mkdir ~/backup
- 2. Copy the rsync script to your backup directory
Download the python script backup.py (click on the name).
- 3. Change the script
There are two major parts that need to be changed in the script backup.py (currently the web backup is not performed, thus I commented the corresponding lines out). Such parts are circled in the following figure.
The upper red circle means the directory in your local mac that you want to back up to. The lower red circle means the directory in the server that you want to back up. In this example I back up my home directory in oas, cabrales. You can change them into your desired directory. Don’t forget to remove/add the for loop in the script if you plan to back up other than this directory, and change the server name accordingly. I use oas in this example (circled in blue). Notice if your user name on your mac is different from the user name on oas, you need to add the oas user name in the blue circle, i.e. from oas ⇒ my_oas_username@oas.
- 4. [Optional] Set up a notification
Disregard this. Growl is no longer freeware.
- 5. Create a cron schedule
- (1) Type “crontab –e” in the terminal window.
The text editor will open a blank window for the “crontab entries” to be entered. Each line presents a separate cron jobs.
- (2) Put the following line in the window and remember to change the directory
* * * * * /Users/YourNameInMacDirectory/backup/backup.py
If you used vi for the edition, when you exit you'll get the message: “crontab: installing new crontab.”
An asterisk (*) is used to indicate that every instance (i.e. every hour, every weekday, etc.) of the particular time period will be used. Here is how positions 1-5 are layed out:
|2||Hour 0-23 (0 = midnight)|
|5||Weekday 0-6 (0 = Sunday)|
- (3) Change asterisks (*) accordingly
Find you name and the time for you to back up in the following table. The starting time means your backup script will automatically run at this particular time. Then change * * * * * according to the corresponding format. For example, my command would look:
40 3 * * * /Users/alejandrocabrales/backup/backup.py
|Fantine Huot||00:00 am||0 0 * * *|
|Joe Jennings||00:20 am||20 0 * * *|
|Taylor Dahlke||00:40 am||40 0 * * *|
|Guillaume Barnier||01:00 am||0 1 * * *|
|Ettore Biondi||01:20 am||20 1 * * *|
|Yinbin Ma||01:40 am||40 1 * * *|
|Jon Claerbout||02:00 am||0 2 * * *|
|Biondo Biondi||02:20 am||20 2 * * *|
|Rahul Sarkar||02:40 am||40 2 * * *|
|Bob Clapp||03:00 am||0 3 * * *|
|Jason Chang||03:20 am||20 3 * * *|
|Alejandro Cabrales||03:40 am||40 3 * * *|
|Huy Le||04:00 am||0 4 * * *|
|Stuart Farris||04:20 am||20 4 * * *|
|Eileen Martin||04:40 am||40 4 * * *|
|Rustam Akhmadiev||05:00 am||0 5 * * *|
|Siyuan Yuan||05:20 am||20 5 * * *|
|Stew Levin||05:40 am||40 5 * * *|
|Shuki Ronen||06:00 am||0 6 * * *|
|Miguel Ferrier||06:20 am||20 6 * * *|
IMPORTANT: Crontab won't work unless your computer is awake.
- 6. [Optional] Check the success of your crontab
Type “ls -ltr /var/mail”, and check the time in the following information
"-rw------- 1 yishen mail 5559 Mar 14 1:40 yishen"
The first backup may take a while, possibly exceeding 20 minutes. Manually running the script for your first backup during the weekend or in the evening instead of using crontab is recommended. To run the script manually, type the following command in your back up directory.
(Ps: probably you will be asked to connect to the server at your first time.)
2. Synchronizing your files and directories from your local computer to your external hard drive
First, Initialize your hard drive
- Click Initialize in the pop-up window when you first attach the hard drive to your computer
- Another window will pop up which is shown as follows. Select a disk which is circled in red;
- Click Partition tab (circled in red);
- Click Volume Scheme to choose how many partitions you need (circled in red);
- Change the Name of your hard disk (circled in red);
- Change the Format into Mac OS case sensitive (circled in red);
- Click Apply (circled in red).
Second, use time machine to back up your local computer
Introduction of time machine
Time Machine is the built-in backup that works with your Mac and an external drive (sold separately) or Time Capsule. Connect the drive; assign it to Time Machine, and start enjoying some peace of mind. Time Machine automatically backs up your entire Mac, including system files, applications, accounts, preferences, music, photos, movies, and documents. But what makes Time Machine different from other backup applications is that it not only keeps a spare copy of every file, it remembers how your system looked on any given day—so you can revisit your Mac as it appeared in the past.
How to use time machine to back up
- Select Time Machine menu > Open Time Machine Preference.
- Click Select Disk
- Choose a drive where backups will be stored, then click Use Backup Disk
- [Optional] In Time Machine preferences you can click the Options button to select items to exclude from the backup
- [Optional] change the backup interval
- (1) From the command line
If you don’t want to install any extra software then you can change Time Machine‘s backup interval from a shell prompt. Start up a Terminal window and then type:
sudo defaults write /System/Library/LaunchDaemons/com.apple.backupd-auto StartInterval -int 18000
The 18000 above is the required backup interval in seconds — five hours in this example. And note that the above command is all on one line.
- (2) Point-and-click
There are several full Mac OS applications out there that allow you to manipulate Time Machine‘s hidden preferences. The two most popular are Time Machine Scheduler (http://www.klieme.com/TimeMachineScheduler.html) and Time Machine Editor (http://timesoftware.free.fr/timemachineeditor/).
How to use time machine to restore
Choose Enter Time Machine from the Time Machine menu and the restore interface appears. You can literally see your windows as they appeared “back in time.”
Encrypt your backup disk
The best way to keep your backups secure is to encrypt your backup disk. Encryption is available for Time Capsule, disks attached to another Mac on your network, and disks partitioned with the GPT partition scheme and attached directly to your Mac.
If you want to change from unencrypted to encypted backups, you must remove your backup disk and then set it up again. Follow these steps:
1. Open Time Machine preferences.
2. Click Select Disk or Add or Remove Backup Disk (if you have multiple backup disks).
3. Select your backup disk, then click Remove Disk.
4. Now set up the disk again as an encrypted backup disk. For instructions, see: http://support.apple.com/kb/PH14110?viewlocale=en_US
The first backup may take a while. You may want to set up Time Machine in the evening so that the initial backup can be done overnight. You should not interrupt the initial backup. You can continue to use your Mac while Time Machine backs up.
Once the initial backup is completed, Time Machine performs subsequent hourly backups of only the files that have changed on your Mac since the last backup (as long as your Mac is awake and the backup drive is connected).
To know more information about time machine, please check the following website: http://support.apple.com/kb/HT1427