UBBDev.com: UBB.classic backup and automation tutorial




Home - Forums - Database - Members - Services - About

Articles
Articles of Interest for the UBB Developer

Reviews
Grab Reviews of all (software, hardware, sites) that's important to you as a developer.

Interviews
Interviews in the world of internet community-building.

Member Spotlight
Grab Honorees for excellence in site design, color scheme, member experience, appropriate use of modifications, and site integration.

Join the Team!
Team UBBDev is our name for the United Devices cancer research project started here at UBBDev.

Join today and donate your computer's unused CPU time to a worthy cause. When you sign up, you'll be helping to search for a cure for cancer, and you'll also get a nifty haxxor icon.

Fig. 1-1: Nifty haxxor icon.

»	UBBDev.com » "U" zine » Articles » UBB.classic backup and automation tutorial

UBBFriend: Email this page to someone!

Author

Topic: UBB.classic backup and automation tutorial

Charles
Moderator Extraordinaire

posted

UBB.classic backup and automation tutorial

INTRODUCTION

Every UBB.classic owner and/or administrator has tried at least once to backup all the content of the bulletin board on their own computer or an other server.

Of course, you can simply fire up your FTP client and download your cgi-bin, noncgi and members folders directly one file by one file. This solution, even thought it works well can take a couple of hours even for someone running on a xDSL line. The goal of this tutorial is to show you an alternative way of doing a backup which is much quicker and can easly be automated.

This tutorial works ONLY for users who run their server under a UNIX/LINUX environment.

You will need a Telnet/SSH account to your server in order to use the following commands but if you don't you should be able to run the full script using a CRON job (I will come to this later).

If you need a telnet/SSH client you can use PUTTY a free client available for all platforms.

PART 1 - Discovering TAR

So what is this magical trick I am going to use in order to backup my UBB.classic in a breeze? Well I am simply going to compress all the files together before downloading/transfer them.

As you know UBB.classic runs on flat files, each file being simple ASCII text contains many characters that take up space (spaces, line break etc.) by compacting everything you can fit a 50-100Mb UBB.classic into a files that is only a few megs.

Assuming you are already logged on your server go to your webdir like this

code:

cd public_html

replace public_html with the name of your dir, it can be www, httpdocs or something else.

then we are going to tar up everything (in this tutorial I use ubbcgi as the cgi-bin folder and ubb as the noncgi folder).
I also assume that the members folder is inside ubbcgi.

We will use the ubb directory as as example, each directory will compacted in its own tar file in order to make retrieval of information easier.

code:

tar -cf backup.noncgi.tar ./ubb

so what does it mean? with c I create a new file, f means that what follows is the name of the file I want to create. Why do I use a dot slash before the 2 directories name? Well dot means the current directory so ./ubbcgi means that ubbcgi is right under the current directory.

lets see what we get:

code:
ls -lh

on my test server my backup tar file is 36 Mb which is still quite big we can do better by compacting the archive a second time thru gZIP.

lets start again:

code:

tar -czf backup.noncgi.tar.gz ./ubb

note that this time I added a z to show that I want to compress the file through gzip and I gave my backup file a tar.gz extension.

again, lets see what we get.
This time my file is only 7.2 Mb, good? We can still do better [Smile]

.

PART 2 - Advanced TAR

The ubb directory contains the cache directory with all the frequently seen pages. these take up space too so the best solution would be to clear the cache before doing a backup but we can't do this since we want an automated solution and also because we don't want to rebuild the cache after each backup.

lets use then the --exclude command.

code:

tar -czf backup.noncgi.tar.gz ./ubb --exclude=./ubb/cache-1234ABCD

without the cache I go down to 4.2 Mb (the 1234ABCD is the code of your cache it is unique for each UBB.classic)

but then without the cache folder I won't have all my statistics, no problem lets just add it.

code:

tar -czf backup.noncgi.tar.gz ./ubb --exclude=./ubb/cache-1234ABCD ./ubb/cache-1234ABCD/ubb_files/counter

THERE! We now have our full forums backed up in a much smaller file and we didn't even have to keep the non necessary files.

PART 3 - Sending the file by FTP

In this part we will see how to send the file by FTP using only one command line and no prompts, off course you could use the FTP command to do it but this way enables more in terms of making our automatic backup script later on.

For this I use a script called ncftp which is often precompiled on the servers, if you don't have it you can find it here: http://www.ncftp.com/ncftp/

here is the command I will use:

code:

ncftpput -u login -p password domain.com /remote/path backup.tar.gz

Please note that the remote path is based on where you arrive with the ftp account so for example if you start in /home/user and want your file to be in /home/user/backup then just enter backup as your remote folder

Even though this works we don't want to have the login and password in the command line because anyone who can see the process will be able to see them. So lets create a config file, remotebackup.conf for example with the following informations:

code:

host remotehost.com
user login
pass password

we can then transfer our backup file using the following command:

code:

ncftpput -f remotebackup.conf remote/path backup.noncgi.tar.gz

Remember to change permissions on the file so no one else can read it:

code:

chmod 400 remotebackup.conf

PART 4 - Creating the actual script

OK, so now we have all the parts we need for our script lets put them together to create our script.

This time we will add one tar line for each of our directories.

Here is a first example:

code:

#UBB.classic backup

#first we initiate the date variable used for naming the files

mydate=$(date -I)

#Lets tar up our cgi files

tar -czf backup.cgi.$date.tar.gz ./ubbcgi --exclude=./ubbcgi/Members #Members are excluded because we will add them in a different file

#Lets tar up our noncgi files

tar -czf backup.noncgi.$date.tar.gz ./ubb --exclude=./ubb/cache-1234ABCD ./ubb/cache-1234ABCD/ubb_files/counter

#Lets tar up our members files

tar -czf backup.members.$date.tar.gz ./ubbcgi/Members

#If your variable is not the same as the cgi folder add an other tar line
#tar -czf backup.var.$date.tar.gz /path/to/var/.

#transfer the files to the backup server

ncftpput -f remotebackup.conf backup backup.cgi.$date.tar.gz backup.noncgi.$date.tar.gz backup.members.$date.tar.gz 

# remove the local files

rm backup.cgi.$date.tar.gz 
rm backup.noncgi.$date.tar.gz 
rm backup.members.$date.tar.gz 

# done!

you will need to transfer this file in ASCII mode and chmod it 755 in order to run it.
To run the file type:

code:

./nameoffile

PART 5 - Making it all automatic - CRON basis

To do this we will use the CRON function. The following is based on ITworld's tutorial on how to use cron.

A cron table file is made up of one line per entry. An entry consists of two categories of data: when to run a command and which command to run.

A line contains six fields, unless it begins with a hash mark (#), which is treated as a comment. The six fields, which must be separated by white space (tabs or spaces), are:

Minute of the hour in which to run (0-59)
Hour of the day in which to run (0-23)
Day of the month (0-31)
Month of the year in which to run (1-12)
Day of the week in which to run (0-6) (0=Sunday)
The command to execute

The first five fields are the "when to run" while the last is the actual command that needs to be run.

An entry in the first five columns can consist of:

A number in the specified range
A range of numbers in the specified range; for example, 2-10
A comma-separated list consisting of individual numbers or ranges of numbers, as in 1,2,3-7,8
An asterisk that stands for all valid values

Note that lists and ranges of numbers must not contain spaces or tabs, which are reserved for separating fields.

A sample cron table file might be displayed with the crontab -l command. The following example includes line numbers to clarify the explanation.

code:

1     $ crontab -l 
2     # DO NOT EDIT THIS FILE 
3     # installed Sat Jul 15 
4     #min    hr   day   mon   weekday  command 
6     30      *     *     *     *       some_command 
7     15,45   1-3   *     *     *       another_command 
8     25      1     *     *     0       sunday_job 
9     45      3     1     *     *       monthly_report 
10    *       15    *     *     *       too_often 
11    0       15    *     *     1-5     better_job 
$

Lines 2 through 4 contain comments and are ignored. Line 6 runs the command some_command at 30 minutes past the hour. Note that the fields for hour, day, month, and weekday were all left with the asterisk; therefore some_command runs at 30 minutes past the hour, every hour of every day.

Line 7 runs the command another_command at 15 and 45 minutes past the hour for hours 1 through 3, namely, 1:15, 1:45, 2:15, 2:45, 3:15, and 3:45 a.m.

Line 8 specifies that sunday_job is to be run at 1:25 a.m., only on Sundays.

Line 9 runs monthly_report at 3:45 a.m. of the first day of each month.

Line 10 is a typical cron table entry error. The user wants to run a task daily at 3 p.m., but has only entered the hour. The asterisk in the minute column causes the job to run once every minute for each minute from 3:00 p.m. through 3:59 p.m.

Line 11 corrects that error and adds weekdays 1 through 5, limiting the job to 3:00 p.m., Monday through Friday.

PART 6 - Making it all automatic - using CRON

Now that we covered the basis of making a CRON file, lets use it to automate the calling of our backup script.

lets say we want to make our backup every day at 11 PM, our cron file will look something like that:

code:

# Automatic UBB.classic backup
0	23	*	*	0-6 /path/to/ubbbackup

Once you have decided when to run your backup just add your file to your crontab like this:

code:

crontab name-of-crontab-file

Note: doing it this way will result in replacing any existing crontab with the one in the file if you are unsure you can first list the content of the crontab using

code:
crontab -l

you can then edit it directly using

code:
crontab -e

but adding extra informations to your file and then adding the file to cron is the easiest way to do it.

Note 2: some servers have some execution time limit for cron, so if your backup script worked when you called it from command line but did not from cron then contact your system administrator to allow for more execution time.

IP: Logged |

Printer-friendly view of this topic

Advertise Here

Accessories
Decorate your desktop with official UBBDev mugs n' mousepads!

Clothing Items
Stay warm this winter with an official UBBDev sweatshirt on your back!