Mysterious Server 2003 disk space consumption
Written by Joe Glessner on October 25, 2009 – 12:00 am -
So the System drive of my (primary) domain controller has been running low on disk space (it’s a 20GB partition running with about 4GB or so free). This has been a nagging issue that I’ve had off and on for a while now, and I haven’t really had the time to delve into it.
I decided to start my investigation by running WinDirStat and looking for any oddly large files. The largest portion of the System disk is consumed by the Program Files directory (no big surprise there), and aside from a couple slightly disturbing large files from my backup software there is only one group of large files on the drive – hovering in at about 12GB for the 8 or so files. And they all have the same path and are similarly named: C:\System Volume Information\{914b4760-84b2-11dd-bca9-000e0cb2b564}{3808876b-c176-4e28-b7ae-04046e6cc752}
Hmmm, interesting. A quick Google search turns up some results linking this directory (more specifically files with CSLID names in this directory) to two things: System Restore points, and virus files.
Well I’m pretty sure it’s not virus files (no other odd behavior or weird network activity), and if I’m not mistaken to enable System Restore on WS2003 you have to manually copy over some files from an XP CD (which is a pretty cool hack, but not something I’ve done on any corporate network I’ve ever worked on).
At this point I start hearing dramatic music in the back of my mind, I’ve got a bonafied mystery! Or at least initial facts would indicate so.
Well a bit more in depth investigation turns up what some of you already knew at this point, the culprit is VSS. But I never configured VSS! (queue swelling of dramatic music in the background)
Ok so this is something of a mystery after all. So I go digging around in the event logs for the last 3 years looking for the initial VSS snapshot message. It sounds like a lot of work, but Microsoft Log Parser actually makes things like this pretty trivial.
Turns out that the VSS snapshots started on the same day that I installed our current Backup software (Yosemite Backup 8.5 sp2) which cooincidentally has the ability to make use of VSS snapshots!
Now this is not a huge issue, as VSS will delete old snapshots when space is needed, however I tend to take exception to software doing things like this without my permission.
Well luckily for me, I used to be a manager at the company that makes our backup software, so I fire up my trusty IM client, and start poking at the engineering department.
Twenty minutes later I have my trusty pipe and smoking jacket firmly in place, as I am feeling quite like Sherlock Holmes. It seems that in fact it was the backup software which enabled VSS for all volumes on my server, and (because it uses the defaults when enabling VSS) had set VSS to not limit the space consumed by snapshots!
A simple trip into Disk Management, and a quick change to the drive’s Property page, and VSS is now limited to 4GB for the system partition (which is far more than I’ll ever need). Interestingly enough had I disabled the VSS service on this machine before installing the backup software, it would not have enabled VSS. I’ve asked that they include a note about VSS being automatically configured to the Yosemite Backup installer (it may exist now, I’m not sure as I haven’t actually read any of the installer screens in years), but who knows when that will make it into the software.
As a side note, I’ve spoken to the Tech Support Manager at Yosemite Technologies (they make Yosemite Backup), and they are currently writing a knowledge base article about this, and how to change the VSS settings from the defaults that Yosemite Backup enables.
Posted in Admin's Arsenal | 6 Comments »
By Jared on Oct 2, 2008 | Reply
Joe, Thanks! This is the exact issue I was trying to figure out since I noticed a drop in disk space a few days ago and after installing Acronis as my new backup software. Didn’t even think to check out VSS. Sure enough that was the problem. Great article.
Click to Reply to This Comment.
By Brainwarrior12 on Oct 2, 2008 | Reply
I ran into the same issue on Vista.
Click to Reply to This Comment.
By LTS on Oct 4, 2008 | Reply
TIMELY!
I was just investigating this issue on our Exchange server. It’s not Yosemite backup, we use Symantec BackUp exec and lo and behold there using a nasty 135GB of disk space are snapshots. It seems that Symantec also performs this nasty little trick.
I blew away all the scheduled tasks for the VSS and that immediately blanked the file so I gained back my space. Now to see how Symantec responds to that.
Click to Reply to This Comment.
Joe Glessner Reply:
October 6th, 2008 at 12:41 pm
BE should only be using the VSS snapshots during actual backups, so it shouldn’t have any adverse effect on your backups.
Click to Reply to This Comment.
LTS Reply:
October 6th, 2008 at 5:11 pm
It does not adversely impact our backups. It adversely impacts my drive space and causes some unnecessary low disk space alarms. The server has no snapshots enabled, so you’d expect that any information left over would be deleted once the backup is done. This is not the case.
I’ll be able to gauge this impact on my other servers tomorrow once I get in the office again.
Click to Reply to This Comment.
By Clay on Oct 15, 2008 | Reply
Great post. I just had this happen on 2 domain controllers. I used the Sysinternals du.exe tool to figure out the space was being consumed by C:\System Volume Information, and your post helped me figure out it was our nightly backups with Backup Exec causing this.
Click to Reply to This Comment.