Dealing with bunglers and prima donnas on your IT security staff

Think your security staffers are trustworthy? Competent? Knowledgeable? Ask a security professional for horror stories and you might think again.

Here’s one from Kevin McDonald, executive vice president at managed services provider Alvaka Networks, a member of the national board of directors of the American Electronics Association and author of several books on cybersecurity. A construction company client of his had a senior IT person who was also in charge of security.

Somehow, this head of security convinced the firm’s owner that it would be cheaper to store the company’s employee databases at his home, where he had fiber-optic lines already installed, rather than store those databases off-site.

You can see this one coming from a mile away: A conflict arose between employee and employer. Before you could say “internal threat,” the head of security was sending threatening e-mails to the construction firm’s customers, telling them that he had their private information.

The action “fundamentally put this guy out of business,” McDonald says, reducing the construction company’s contracts by some 70 per cnet. It took six months to shut the rogue employee down, given that – of course – he was an authorized user.

Only when the employee threatened, publicly, online, to use the data in an illicit manner, was the FBI in Los Angeles able to enter the employee’s home – after the employee had already built a site and lain plans to put some hurt on his former employer.

It’s a worst-case security scenario of hiring a nut case. Unfortunately, the security sector isn’t immune from bozos, incompetents or know-nothings, whether in their midst or passing down decrees from above. Indeed, security pros are less likely to be judged on the merits of their output than are, for example, code jockeys.

What gets in the way can be politics, bad luck, misguided C-level execs, out-of-control consultants, lack of communication, isolation from other parts of the business, blind faith in certifications or simply the difficulty of getting rewarded for what doesn’t blow up.

And that’s just a partial list.

But take heart. Good companies can weather bad apples in security. Herein, an outline of common security weak sisters, along with the tools on how to cut them off at the knees.

Bundled badness

At this moment somewhere in corporate America, security staffers are cursing their C-level execs for foisting on them bundled junk. Here’s how it works: Salespeople from the big guys — be it Symantec, Trend Micro, McAfee or CA — come in and propose to a C-level executive that, for an entire organization, they’ll provide a package that does desktop antivirus, e-mail security, intrusion detection and Web filtering, all for $38 per seat.

What’s wrong with that picture?

“At that point, you’ve commoditized those critical parts of the security infrastructure,” says the head of a security software vendor who requested anonymity. “The problem is, the perception of C-level execs is that security is a commodity. One is the same as the other.”

It’s not that those vendors aren’t good. It’s just that they’re not good at everything. Symantec AV has a stellar reputation, for example, but some security professionals consider its antispam functionality to be less than best of breed.

Repercussions from one vendor’s successful sale into the U.S. Department of Defense are still being felt. “[This particular vendor] won the DoD contract. Then we start hearing from guys at various DoD installations saying ‘Oh, God, this is horrible stuff. We can’t use this,'” says the anonymous vendor.

Organizations whose C-level execs buy bundles do save money — lots of it. Unfortunately, they often get “really subpar security; sometimes dangerously so,” says the vendor.

But how to get that through the head of the C-level exec who’s sold on a bundle? By getting security personnel in on the decision-making process, before the money has a chance to drift out of the C-exec’s hot little hands.

Bob Maley’s lucky that way — his employer fixed the problem shortly before he came on board. Before he took on the job of chief information security officer for the Commonwealth of Pennsylvania in late 2005, the Commonwealth had developed an enterprise architecture process patterned after that of the National Association of State Chief Information Officers (NASCIO).

Part of that process, now in place for some more than four years, is a clear set of standards for security product selection.

As Maley puts it, some other parts of the government may have unlimited resources to purchase security tools, but not his. So he and his group have gotten good at collaborating with peers — not only through NASCIO but also through the Multi-State Information Sharing and Analysis Center (MS-ISAC).

Under the MS-ISAC, which is run through the U.S. Department of Homeland Security, all 50 states share best practices. As well, the organization recently has hitched a ride on the federal government’s SmartBUY purchasing initiative, designed to leverage the government’s hefty buying clout to save money through aggregate purchasing. What works for one sector — the government — in this case works for others: Network with peers, find out what security tools they use and trust, and find out which are clunkers to avoid.

But if it’s not an option to cut your bundle-buying C-exec out of the picture altogether, salvation comes down to intervention at an early stage. Communication is key, and not the type of communication where security says “We have to use XYZ because I said so.” Rather, security has to convert the geek discussion into a business discussion.

“I recommend that security get users to buy into them as people,” advises Alvaka Networks’ McDonald. “Do lunch and learn internally. Bring staff in, bring management in and have them understand why the things you’re saying are being said.”

That helps security pros to break down the “You’re just in the way” barrier, McDonald says. “If you ask the employees and management,

‘So, I have these things I’m being told I have to do — say, to secure PCI information, or to protect assets of the organization, and do other things mandated by government. What would you have me do if you were in my seat?'”

It’s not formal training; rather, it’s getting together and figuring out how to do the security task at hand.

Useless certifications

Security also suffers from paper tigers. “We hire guys with wonderful degrees who are just idiots,” says one security vendor who requested anonymity. “We’ve had guys in here who’ve got degrees and certifications and they can’t even wire a network. They know the words, but they don’t know how to sing the song.”

“For years now, people were getting certifications left and right,” Maley agrees. “They might have five different acronyms after their name. … Honestly, [in] the certification industry, there are brain-dump sites. People can get certified without having experience.”

Maley says that from what he can tell, hiring managers see the acronyms, get impressed and let extensive vetting slide. To avoid hiring paper tigers, employers have to look at a resume and then map the experience back to the listed certifications, he says.

That said, Maley would hire CISSPs (Certified Information Systems Security Professionals), CISAs (Certified Information Systems Auditors) or CISMs (Certified Information Security Managers) — if he could afford them, that is.

“CISSP, I wish I could say I’m hiring them,” Maley says. “I can’t pay those guys enough.” As far as CISAs or CISMs go, Maley says that typically CISSPs have those certifications, which reflect what he calls built-in experience. “You can’t get those unless you show you have that experience,” he says.

Getting what you pay for

Speaking of not being able to afford CISSPs, Maley says that not being able to afford qualified security staff has been “one of his biggest challenges” in heading up cybersecurity for state government. In fact, Maley estimates that there’s a pay differential of anywhere from 20% to 100% between the public and private sectors.

“I lost a gentleman who doubled his salary when he went to the private sector,” Maley says. “For me to get a security expert in, even if I would take them up to the highest step in their pay category, it doesn’t come close to what they could get in the private sector.” And Maley can’t entertain the notion that a given hire will stay with him for the long haul.

What he does to get around having an inexperienced security staff is to hire those who are “a little wet behind the ears” — sometimes right out of college — but who show promise.

The lure for such hires is the chance to work in an enterprise environment where security staffers have the chance to spot cyberattacks as they hatch. In the past six months, for example, his security team has seen three variants of the Storm Trojan come in that hadn’t been spotted elsewhere. That’s not surprising, given Symantec’s April 8 Security Threat Report (download PDF), which cites a shift in attacks aimed at sites that are likely to be trusted by end users, such as social networking or government sites.

“I’ve got a team that has the opportunity to fight that kind of stuff, analyze it and be on the leading edge in the fight between the bad guys and us,” he says. Recruits get hands-on experience on projects that are both significant and “exciting,” including a penetration-testing rollout partially automated with Core Security technology in response to repeated interruptions from virus outbreaks, Maley says.

Maley also coaches his green recruits at building their resumes. He knows that eventually they’ll leave, but if they’re adding to their resumes, having fun and learning in the meantime, chances are they’ll stay that much longer — a trick that any revenue-challenged organization can employ to good effect.

Skirting the underskilled

When dealing with a security staffer with limited skills, you’ve got to limit his potential to blow everything up. This approach is called “putting a skirt on him” by the don’t-quote-me crowd, but there’s a more positive spin to put on it. Anthony Scalzitti, a security engineer at a major security software company, says it’s all about limiting potential mistakes by assigning tasks on less critical systems — for example, investigating suspicious log activity or IDS reports.

Another useful security role that won’t get a limited-skill staffer into trouble is to attend meetings of other business teams to make sure the security group is aware of upcoming projects. Having a security representative sit in on team meetings can also help to remind colleagues to build security in from the design phase instead of shoe-horning it in after design and development.

“Take a newer [security staffer], or a younger one, to be that person,” Scalzitti says. “Even if they don’t contribute a lot, if they’re in the meeting, those people say ‘Oh, we have security here,’ and they feel obliged to think about security. The person may not contribute a lot, but that’s when a more experienced part of the [security] team tells them what to say next.”

As it is, many organizations have struggled to integrate security as an element of quality in application development, alongside speed, failure resistance, scalability and the need to meet business requirements.

Having a warm security body on hand can thus serve not only to educate the security newbie and keep him out of trouble, but also to get security’s voice heard.

“These are useful roles, and mistakes generally don’t impact business,” Scalzitti says.

Deflating prima donnas

Security prima donnas are the opposite of security boobs, but they’re still a pain to work with. These are the staffers who regard certain tasks as unworthy of their time, including reviewing logs or activity alerts, doing simple configuration reviews or meeting with other business groups.

In handling such divas, Scalzitti has had success putting them to work researching security incidents that appear in the media. The point, he says, is to get the security elite to discover that 80% of incidents are a result of simple attacks on low-hanging fruit.

“In information security, there are so many opportunities for an attacking hacker to pick a company,” he says. “Unless they [have a grudge against a particular] company, they’re going to go for low-hanging fruit. Having [prima donnas] research low-hanging fruit, it may take some time, but they come to realize the basics of how things happen.”

The last resort

It’s good to have tools to deal with security’s bad apples, but one ounce of prevention is worth a pound of cure. Many organizations have a 90-day probation period policy for new hires.

Once past the 90 days, most states make it difficult to dismiss an employee without jumping through hoops to establish cause. The lesson: Watch new security employees like a hawk during their first 90 days in order to avoid getting stuck with security flunkees.

Share on LinkedIn Share with Google+