Thank Goodness for Backups – Part 1

Today was a slow day, which meant that something big was building up to happen. And it was a big one. A really big one. That was only spotted by accident.

Our network storage has been getting low recently, and although my colleague and myself have been attempting to shrink down the side of some files, we can’t easily keep up when 10GB of data is added in a week. To save time reprocessing the same files each time, it was decided to only look for files that have been modified in the last 7 days. After a bit of Googleing, I managed to find a nice quick PowerShell one liner that I modified slightly (replace <drive> with the relevant letter):

After letting the command run (and sending the output into a text file) we had a nice 45MB file. This seemed a little bit excessive, and opening the file it showed it contained 327192 lines! Something definitely wasn’t normal here.

A list of files that have been modified in the last week
Turns out I’ve inadvertently written 1/9th of Wolfram|Alpha. Oops.

A large number of lines appeared to end in a “exx“, which we initially though were the files for a game of some sort. However, as they were all in different students folders, this didn’t tally quite right. Looking further through the file, there were also a number of other files which didn’t seem as though they should be there too…

A list of text files that shouldn't be in these locations
I’m not sure about you, but I don’t remember Microsoft saying that they were going to be putting a lot of files called “HELP_RESTORE_FILES_<letters>.TXT” inside the roaming AppData folders

Now having a slightly bad feeling about what may have happened, I located one of these “HELP_RESTORE_FILES_<letters>.TXT” files (I had a few hundred to choose from) and opened it to see what exciting contents it would hold.

The contents of one of the "HELP_RESTORE_FILES_<letters>.TXT" files
I’m not quite sure why they’re telling me to avoid missprints – they must’ve received bad service in the past.

Barring English language quite no good, and speeling mistaks ocasional use, the outcome of this file did not bode well at all. Searching the first line that is written, “All your documents, photos, databases and other important files have been encrypted with strongest encryption RSA-2048 key generated for this computer“, revealed that it was CryptoLocker, or a variant of it, that had been making its way through a number of our files.

The first thing that was needed to be done was to see who had caused this to happen. One of the good things about computers is that they log everything, so it’s easy to point the blame, and hard to hide from it (unless you’re in charge of the logs). Right-clicking on one of these files and going to Properties > Security > Advanced > Owner showed the name of the staff member who had accidentally created all of them.

The Windows property window showing the owner of the files
Remember, everything is logged

I called them from the office phone and asked them to come over straight away with their laptop and any memory sticks. At least this would stop the program spreading if it was still on their devices, and I knew that the backups of the network would at least get the other files back, which is why I didn’t start on recovering them straight away.

They came into our office and understandably were confused and wondered why they had been called over. We asked them if they had been opening any files or browsing any websites around 10:30 the previous day, and they said that one website they went on to looked dodgy, and so they immediately clicked the browser back button. This was fair enough, and as far as they were concerned, nothing had happened.

We showed them the files that had been encrypted, and asked if we could have their laptop and memory sticks to check that they were fine. At this point, the staff member involved was quite worried about what would happen to all the files and themselves, so my colleague and I spent the rest of the time they were in our office reassuring them that everything was going to be fine and that they weren’t in trouble in any way at all (I’m too nice to tell people off if they do something wrong anyway), but we just wanted to make sure that everything was fine now.

Booting the laptop from a USB drive to avoid the normal Windows boot up, I browsed to a few locations and saw that there were a number of “exx” and “HELP_RESTORE_FILES_<letters>.TXT” files, but their memory stick was fine. They could have their stick back, but the laptop was now not trusted and had to be reimaged. After yet more confidence boosting from myself (probably not that good) for the staff member, we said that we’d be able to return the laptop in a few hours, that I’d start restoring the files now and then let them know that it was all back and working as though nothing had happened.

The first step involved seeing exactly how many files had been encrypted on the network, and where they were stored. As CryptoLocker only works on local drives and mapped network drives, this narrowed down the amount of searching that needed to be done. We knew that the local disk was already damaged and being reimaged, so now it was just a case of seeing what network drives that they have access to, and had the files been encrypted.

Opening the command prompt and typing the below line at each mapped drive root folder gave me an indication of how many needed to be restored.

A list of some of the files that had been encrypted

It was a large number, but even though the encryption had been happening for around 6 hours before the laptop was turned off, it had only managed to get through the local hard drive, the users home folder and halfway through the little peoples home directories, which was around 19.3GB in total. Considering that our daily nightly backups come to about 2.7TB in total, only loosing just over four DVD’s worth of data (while bad) is nowhere near as damaging as it could have been.

I fired up our backup software, and now that I had a list of all of the folders that needed to be restored from the output of the command above, I set about starting to get them back.

The files being restored by the backup software

All that I can say now is thank goodness for backups. And accidental discoveries.

Leave a Reply

Your email address will not be published. Required fields are marked *