3
|
|
3 |
|
|
The LA Fox Developer Newsletter
|
|
July 1999
|
|
It seems like I hear about data loss disasters either first-hand or second-hand almost every month. While everyone pays lip service to the idea of backing up their data, I have found that very few companies really do it well. Here are just a few of the cases I have seen lately.
Case #1:
A small hospital in Nebraska suffered a Novell server crash. When the server crashed, their FoxPro DBF files were corrupted. When they attempted to do a restore, the backup software truncated their existing data files. The data tapes were sent to a data recovery house which informed them that the tapes were blank.
What
had happened was the hospital had replaced their old tape drive unit and software with a newer one. Unfortunately, no one had ever tested doing a restore with the new unit. Fortunately for the hospital, they still had the old tape drive unit and backup tapes. As a result of this failure, this hospital lost 6 weeks worth of data from their system. The cost in terms of labor was probably in the tens of thousands of dollars to re-enter the data.
Lessons Learped
1) Before doing anything, back up the data that you have left onto at least
Z
meala
Defore
trying
to repair or restore the data Preferably, zip the data up on to another hard drive.
2) Periodically test doing a restore. If possible, test a complete restore of the data.
3) Keep old backup units, software and tapes around for a while. Don’t just throw them out the minute they are replaced.
Case #2
A computer manufacturer called
in
a panic. Whenever they tried to access their FoxPro application they got the infamous File xxxxx.dbf is not a FoxPro DBF message. My initial thought was to run a header repair utility and fix the file. However, when I looked at the file it showed a length of 0 bytes. The customer had been having trouble with a RAID array on the server and it looked like it had been the culprit. At this point the customer wasn’t worried. He thought we could restore the data from tape. When I looked at the tapes, I found that the application data had not been backed up because the files had been held open by the application. One of the users, not knowing any differently had always left the application running on his desktop. When the backup program ran, the files could not be backed up. The customer was also using an extremely poor tape rotation system. Essentially, he only had
two
weeks of data. Fortunately for that customer, someone had made a copy of the data 6 weeks earlier and stored it on a zip drive.
Lessons Learned
1) Review your backup logs and make sure your data is really getting backed up. If possible automate the log scanning
|
process and have the results sent via email to the administrator. 2) If your backup software doesn’t support backing up open files, you may want to investigate software that can.
3) Use a good tape rotation method. Saving a few hundred dollars on tape purchases isn’t worth the risk of losing data. The rotation I recommend is this:
Week # I
Monday Differential Backup to Daily Tape
—
Set 1
Tuesday Differential Backup to Daily Tape
—
Set I
Wednesday Differential Backup to Daily Tape
—
Set I
Thursday Differential Backup to Daily Tape
—
Set I
Friday Complete Backup to Weekly Tape
Week#2
Monday Differential Backup to Daily Tape
—
Set 2
Tuesday Differential Backup to Daily Tape
—
Set 2
Wednesday Differential Backup to Daily Tape
—
Set 2
Thursday Differential Backup to Daily Tape
—
Set 2
Friday Weekly Tape
Week # 3
Monday
—
Thursday Differential Backup Overwrite Daily Set I
Friday Weekly Tape
Week # 4
Monday
—
Thursday Differential Backup Overwrite Daily set 2
Friday Weekly Tape
You should have at least 5 Weekly tapes in the rotation. Once you have 5 weekly tapes, overwrite the oldest weekly tape on the next weekly backup. The last weekly tape of each month should be saved for at least one year. You might want to just save the last weekiy
backup of
each month forever.
Using this rotation method the total # of tapes required for one year is 25 tapes. Once started, this rotation only requires purchasing one additional backup tape per month.
Case#3
A customer called and said that their RAID array had gone down, crashing their SQL database. Fortunately for the customer, the system went down 10 minutes after the last backup had ran. If the array had crashed 6 hours after backup, the results could have been much worse.
Lessons Learned
1) Don’t believe that redundancy features in hardware will protect you from data loss. As often as not, data loss is caused by a failure in the application software or Network Operating System. In this case, their
redundancy
hardware just plain failed.
2) If you are doing disk dumps of a SQL database, make sure that you get those dumps on tape as fast as possible. Alternatively, do your disk dumps to a separate physical drive.
3) If your SQL database supports transaction logs, make sure those transaction logs are stored on a different physical device.
Case#4
This is one I heard from Jim Slater of the RMFUG. The story goes that a customer’s Novell server crashed and the hard drive
|
|
Page 3
|
|
|
3 |
|