Using Shadow Copy to save the day from the beach. (reRun)
Check out my side of this panicked phone call from the early morning junior admin.
“What do you mean the backup did not run last night?”
“Why exactly wouldn’t the tape drive be plugged into the same UPS as the library?”
“His WHAT was plugged into the ups? His cell phone? Why wouldn’t you move it?”
“Yeah, yeah yeah, Of course I can get your files back I have plan 1-di0-t in place.”
I checked the file server and someone definitely deleted moved or otherwise did something to our main executive share. The deleted files were already mirrored to the raid mirror and via my xcop
y script this morning at 6am.
In about 10 minutes it will be 8am est and shit WILL hit the fan.
Did I mention I am not in the office and in Costa Rica? (business not pleasure kids! I’m building wireless infrastructure. Don’t be jealous.)
So from the beach (OK, you can be a little jealous.) I was able to remote desktop into my file server via my favorite HP 2710p Tablet. (It works in the bright sunny light!) Yup everything was gone and there is no network recycle bin.
So what did I do? How did I fix it quickly?
I restored the whole thing from Shadow Copy in 10 minutes flat.
Check out the steps I used:
I restored the whole thing from Shadow Copy in 10 minutes flat.
Check out the steps I used:
First the server needs to be running 2003 or better.
Next the box needs to have shadow copy enabled on the volume with your files. Capacity is so cheap, that I have it enabled on all my servers volumes backing up 5 times a day.
Now that gives me access to 5 revisions a day going back as long as space permits. You can modify how much space Shadow Copy uses. Remember it is WAY better to be proactive and over prepared
for these times than not at all. It is the difference between easy work and pulling your hair out of your head.
In this instance I opened the share right clicked choose properties and then previous versions.
I saw this:
Each of those dates and times are full folders from the specific time. Apparently my shadow copy attributes have been modified and I only have 2 copies per day… But onward and upward…
I clicked on this mornings earliest backup but the files were not there. Onto yesterdays last backup around 12pm and bingo bango I had my files I dragged the folder back to its place on the volume and was about to call it a day when I decided to find out what happened. Or at least know who did it when and if it happens again in the future.
I enabled auditing and I will show you how when I get back!
Should the junior admin be fired for not correcting the tape drive issue? How about the night admin who’s cell phone was plugged in instead of my tape drive?
Now let me get back to tanning…working! Today’s challenge of the day:
Getting the damn monkeys to stop eating my wires and flucking with the repeaters.
Yes Admins rule the world and can do it from the beach!
_TheEnjoyYourJobAdmiN_
April 4, 2008 - 10:54 am
Haha, that was a very funny post. Not to mention a great strategy in using Shadow Copy, I will have to check that out. Get back to "work" :D
Click to Reply to This Comment.
April 4, 2008 - 11:48 am
Thanks Jim and all the other well wishers! If you can give this post a thumbs up on Stumble Upon for us! :)
Click to Reply to This Comment.
April 4, 2008 - 3:21 pm
Shadow copy is probably one of the best things M$ every came up with. It has saved me hundreds of times.
Click to Reply to This Comment.
April 5, 2008 - 5:52 am
Yup! and did you hear they extended the xp life till 7 is released? And they say watch out for 7 (per Bill Gates) before 2009!
How is everything going buddy?
Click to Reply to This Comment.
April 7, 2008 - 11:14 am
Shadow Copy rocks! I have an employee who periodically "forgets" and overwrites their documents instead of doing a "save as" when they create a new one.
Shadow copy saves me a ton of time. I used to have to go to the backup tape every time they did it. Now I restore the shadow copy and the whole job is done in 2 minutes.
Click to Reply to This Comment.
August 18, 2008 - 8:59 am
Shadow copy has saved my bacon more times than I care to admit.
Click to Reply to This Comment.
August 18, 2008 - 9:17 am
It amazes me how many admins have never even heard of SC. I’ve had to explain it many times. Have they been living under a rock? I can’t even remember the last time I had to go to tape to recover files (other than testing my backups). SC gives you up to 64 versions. I would personally rather be able to go back 64 business days so I only do 1 copy a day. I think it depends on the type of business you support. My primary goal is to never have to depend on tapes again.
As for your two admins, I think it depends on whether they realize how dumb their decisions were. Purposely unplugging a work-related device to plug in your cell is about as dumb as it gets. Technically, nothing was lost so I would drop it as long as they both learned a valuable lesson. I know my strong feelings about backups were created because data was lost, luckily not by me personally. There is no lamer feeling than telling a user, regardless of whether it’s a secretary or CEO, that the document they worked on for 6 months is gone – because you didn’t do your job.
Click to Reply to This Comment.
Karl L. Gechlik | AskTheAdmin.com Reply:
August 18th, 2008 at 9:28 am
The Kid got fired but if he had a better attitude when approached he might not have. If you do something wrong fess up and own up to the responisibily – I don’t want to hear well nothing happened so no harm no foul.
BE apologetic when you fluck up for everyone’s sake! I also can’t look like a whimp/punk in front of my team. They would have lost all respect for me!
Click to Reply to This Comment.
August 19, 2008 - 11:40 am
So you fired the night admin, right? he is the one at fault here. The junior admin should only have gotten it if he had some how known (monitoring maybe?) that the tape drive was unplugged and nothing was done about it.
If he got lippy with you, well that’s what the “clue-by-four” is sitting in the server room for.
Unplugging a Tape drive (of all things) to plug in a cell phone is about the most egregious mistake you could make as an admin (well maybe storing the electromagnet on the backup tape vault would be worse, but it would be a close call).
Click to Reply to This Comment.
Karl L. Gechlik | AskTheAdmin.com Reply:
August 19th, 2008 at 12:51 pm
Yes I fired the Night Admin and the Junior Admin got a slap on the wrist. The Junior Admin DID see the phone on the ups and did not re-act or do anything but he is severely underpaid :)
Click to Reply to This Comment.
Mike R. Reply:
August 20th, 2008 at 8:44 am
Hehehe I used a electromagnet on some backup tapes as insurance before I left my last job. What a buncha pricks!
Click to Reply to This Comment.
August 21, 2008 - 9:46 am
good post, thank you for this!
Click to Reply to This Comment.