Back in January, Scandinavian gamers hijacked a New Hampshire medical center's server to host "Call of Duty: Black Ops" sessions. When asked about that incident, Stephen Heaslip of the gamer site Blues News said hackers are not the most likely individuals to commandeer corporate servers for illicit gaming: Such appropriations are more often the work of IT administrators. When asked if he could put us in touch with some of these rogue game server admins, Heaslip posted a call to his readership - and four volunteers stepped forward.
We'll call them Mr. North, who is director of network operations for a midsize manufacturing company; Mr. South, an IT administrator in the poultry business; Mr. East, a university systems admin when he was active in this realm; and, Mr. West, a senior systems admin in the medical industry. Here's what they had to say:
How common is this kind of activity within IT departments?
Mr. North: It is very common to see this kind of stuff going on. As long as the users don't notice something like slow connection speeds or not being able to get their e-mail, no one really bothers us.
Mr. East: I hadn't really seen it discussed until this topic came up on Blues News, but it seemed apparent then that most of the old faces I'd seen posting (on that site) for years had also done the same things.
Mr. West: I would say it is rather commonplace. Obviously at different orders of magnitude depending on how strict management is and the awareness level of people who aren't in on it.
Describe some of the games that you've hosted on company equipment?
Mr. South: I hosted a 24-slot Counter-Strike: Source on a company T-1 for about three years. I brought in my own server and put it under my desk and ran it that way. The only company equipment involved was the switch I plugged into and the router that hit the net. I also hosted a 20-person TF2 server for two years during the same period. This was hosted on a decommissioned server that the company wasn't using for anything. ... We mainly played at night. I don't recall any significant activity during the day.
Biggest insider threat? Sys admin gone rogue
Mr. North: Currently I have "test realm" for World of Warcraft running that we use to test out gear and specs before we commit to doing so with the actual pay version. I have a Red Hat system that is just used for DNS and mysql server that we are hosting the WoW server and vent server on.
Mr. West: In the past we've had Team Fortress 2, Killing Floor, Counter Strike, Minecraft, and a few others. We've actually run the servers off of a few different boxes. As the company grew/changed we'd need to switch things over to a different box so as not to overload a production box with non-production processes. Obviously it's in our best interest to not cause downtime or other issues so as to not draw attention.
What are the primary motivations for doing this stuff? Saving money?
Mr. North: Really it's about two things: The cost savings of hosting our own vent server alone is worth it, but also it's a learning experience for the techs; they have to maintain security at all times on the network as well as load balancing and QoS to allow this to run as smooth as possible.
Mr. West: My motive is to have a free server for myself and my group of friends. We essentially have full control of the box including creating users, running services, compiling code, etc. If we didn't have the free server I highly doubt we'd have one at all. Half of the fun is in flying under the radar.
Mr. East: A lot of it was "because I could."
How much do you worry about getting caught?
Mr. South: I didn't really worry. I wasn't using bandwidth during peak hours, and I was on great terms with my boss (the CFO).
Mr. West: It is a mild concern, but by and large such things are allowed (tolerated?) with a wink and a nod. There's also an understanding that the games will not have an adverse effect on business. ... It's hard to get caught when you're the one in charge of the servers and no one else looks at them.
Mr. North: I never worry; I mean that's why we are hired is because no one else can do what we do, and anyone smart enough to find out should come and talk to me about a job!
Did you ever have any close calls where you almost got caught?
Mr. North: Yes, it was the result of an office prank where someone attached speakers to a tech's workstation and had them on full. I had the owner of the company in my office and the tech alt tabbed back into a game, which alerted the boss that something was going on. As he got up to go look, I had used VNC to shut down the workstation. I blamed the noise on a PC that was going bad and said that it did that from time to time, which resulted in money to upgrade our workstations. So it was close call and a blessing all at once.
Mr. West: We popped the breaking in the rack, causing a service-wide outage for about an hour or so. One of the members in the group had acquired a high-power server that would be capable of running dozens of VMs at a time. He offered to let the group use it provided it could be put in the rack with the rest of the servers. We didn't think any harm would come as it would replace the current box we were using.
After plugging the server in and letting it run for a few days, all seemed well. That was until we actually started adding VMs to the machine. The extra VMs increased the load, which increased the power usage, which overloaded the circuit breaker in the rack and brought it down.
Mr. East: There was never a mention of the game server for the best part of three years, and one day during a staff meeting, I referred to the server by name and my boss said, "Is that the one with the game server on it?" I still have no idea if he was joking, and he certainly didn't care if he wasn't. It was never mentioned again.
6 tips for guarding against rogue sys admins
Why do you think it's OK to do this?
Mr. South: I never really thought about it in terms of right and wrong. I used company resources that were not being used by the company to build and maintain a community of gamers. I spent lots of time in my office, almost an unhealthy amount. I just saw this as an unspoken benefit of my job.
Mr. North: The way I see it, we keep the network running in tip-top shape, we get the job done and no one really ever complains, so why not reward my techs by allowing them to do this? Other people who do well at my work get company cars and different perks, but not us in the IT department, so this is my way of keeping my techs happy.
Also the equipment is never in use (for business purposes) after 5:30 and on weekends, and since we are paying for the bandwidth, we might as well make use of it.
Read more about data center in Network World's Data Center section.