When your network admin hijacks your system
- Date: July 16th, 2008
- Author: Tricia Liebert
- Category: Attack, Data Security, IT Security, Network, Privacy, Responsibility, Support Positions, Violation Of Trust
- Tags: Network Administration, Network, Terry Childs, E-mail, Team Management, Leadership, Productivity, Networking, Security, Online Communications
The situation has finally happened. A network admin has brought a city to its knees by changing the master system password. What IT leadership needs to learn from this situation. And what they need to do to cope.
______________________________________________________________________________
Hollywood, the nice people who taught us that if you blindly pound the keys on your keyboard actual commands will magically appear on the screen, never saw this coming. Or maybe they did and didn’t want to scare us with actual fact. Now it has happened.
Terry Childs, a 43 year old network administrator for San Francisco’s Department of Telecommunications and Information Services is currently in jail and being held on $5 Million bail. He is accused of altering the city’s FiberWAN network system to deny service to authorized users and setting up devices that would allow unauthorized service to the system.
I hope that our Security blogger, Chad Perrin will speak to the security issues in this case. My focus is on the leadership issues that arise from this situation.
What we currently know is that Childs has been employed by the city for five years. When he was hired he disclosed that he is a convicted felon- aggravated robbery and aggravated burglary in 1982. He was on probation or parole until 1987. According to a city official, Childs had recently been disciplined on the job for poor performance and was potentially to be fired.
It is believed that Childs began tampering with the system around June 20. He set a master password that cannot be overridden and does not allow for sufficient access to upgrade or maintain the system. Further, it appears that he may have enabled a third party to access data from the system that houses the 311 system, the city e-mail system, and the city servers, including confidential information. It also appears that he was reading his bosses e-mail.
I am so glad to not be this guy’s manager today. Even happier that I am not the manager’s manager or the person who hired this guy.
The city has brought in Cisco Systems to help them to get back in control of their brand new $3M system. They think that the bill to hack back the system may run into the millions. And Childs is being held on four felony charges. But how did it get this far?
Apparently the city hired a new head of security some months ago. She began auditing who had password access to the system. Childs seemed to not handle this well and began photographing her. His behavior became increasingly possessive of the system but he continued to have access to it. In addition, he had access to his bosses emails regarding his conduct. At least until he was taken into custody on July 13. But why did it take so long?
It is difficult to see the whole picture as events are unfolding. In hindsight, I am sure that we all agree that this guy should have been assigned to work with another system or simply put on administrative leave until the mess was sorted out. That never happened.
As the leader in this IT group, how do you go about insuring that the problem cannot happen again? How do you keep it from happening in the first place?
Childs’ behavior had been increasingly odd since June 20. To the city’s credit, an investigation of Childs had been undertaken but it appears that no one thought to limit his access to the system. I would have thought that would be the first step.
In the same situation, I would have taken the time to pull the employee off the system and at the least, had a chat about what was bothering him to the point that he felt it necessary to photograph his seniors and monitor their email. But there must have been earlier warning signs that were overlooked.
Regardless of what was missed and what should have been done differently, leadership has a difficult task in front of it. Beyond all the additional workload that is being shunted to Cisco, leadership needs to consider the other admins who are doing the same or similar job and consider how they can avoid a repeat of the situation. That will mean some discussion with the employees but in a manner that is not threatening to them. Certainly it will mean new oversight and new processes.
Normally, I would work with the Administrator team to define what the new processes should look like and incorporate the ideas into a new plan that we can all feel comfortable with. But given the situation, I should reconsider how much input I should take from the team. After all, a member of the team caused this problem. On the other hand, I can’t blame the whole team for the actions of a rogue employee.
What would you do as the manager of this department, given this situation? Is there any “right” answer? Or is this situation truly breaking new ground?
More information:
SF Officials Locked out of computer network (San Francisco Chronicle)
Computer Engineer keeping quiet on lockout (San Francisco Chronicle)
