Heartland CEO on data breach: QSAs let us down

For Heartland Payment Systems Inc. CEO Robert Carr, the year did not start off well, to say the least.

In January, the Princeton, N.J.-based provider of credit and debit processing, payment and check management services was forced to acknowledge it had been the target of a data breach -- in hindsight, possibly the largest to date with 100 million credit and debit cards exposed to fraud.

In the following Q&A, Carr opens up about his company's data security breach. He explains how, in his opinion, PCI compliance auditors failed the company, how informing customers of the breach before the media had a chance to was the best response, and how other companies can avoid the pain Heartland has experienced.

Take us back to the moment you were told a breach may have happened. What was your first thought?

Carr: "It was a Monday night in January, just after dinner, when I was told data files were found on our servers that were not created by Heartland. That was a clear sign of trouble. It was a sleepless night. The question people always ask is what keeps me awake at night. Well, this is it."

What have you learned in recent months regarding how exactly the burglars were able to get in? What have investigators flagged in terms of the big security holes that were exploited?

Carr: "The audits done by our QSAs (Qualified Security Assessors) were of no value whatsoever. To the extent that they were telling us we were secure beforehand, that we were PCI compliant, was a major problem. The QSAs in our shop didn't even know this was a common attack vector being used against other companies. We learned that 300 other companies had been attacked by the same malware. I thought, 'You've got to be kidding me.' That people would know the exact attack vector and not tell major players in the industry is unthinkable to me. I still can't reconcile that."

How did the QSAs respond when you expressed this view?

Carr: "In the post-Enron environment, the auditors have contracts with clients that essentially absolve them of gross negligence. The false reports we got for 6 years, we have no recourse. No grounds for litigation. That was a stunning thing to learn. In fairness to QSAs, their job is very difficult, but up until this point, we certainly didn't understand the limitations of PCI and the entire assessment process. PCI compliance doesn't mean secure. We and others were declared PCI compliant shortly before the intrusions."

Do you see PCI DSS as an ineffective waste of effort, or is this a case where the standard was fine and the audits were off?

Carr: "If a smart person's job is to define a set of rules to keep merchants from being breached and they have to start somewhere, what they come up with is going to look something like PCI. There has to be a lowest-common-denominator set of rules. PCI could be improved, but the standard is fine. The problem is a system where you have a magnetic stripe that's exposed, the number is very valuable, and you can easily buy sniffer software off the shelf. Immediately after the Hannaford Supermarkets breach, where we learned a sniffer had been used, that was a whole new paradigm. That's when we started working on end-to-end encryption. Data-at-rest encryption was no longer enough. Data in transit can be captured."

Page Break

You've no doubt moved aggressively to improve security. Talk about the specifics of what you've done in terms of technology and people policies.

Carr: "Four different card brands have their policies and ideas about security, and we've done everything asked of us. We must have more layers than anyone out there. Some specifics: We re-imaged all our servers -- nuked them, essentially -- and started over. We added additional network segmentation, much more intense monitoring, and added data loss prevention technology, specifically Symantec's Vontu product, which helps you find every place where a card number is stored."

How much money has Heartland had to spend to address the security holes and other things like lawsuits?Carr: "In the first half of 2009, we laid out $32 million and we don't know what will happen going forward. We are aggressively defending against litigation. That's all I can say."

How receptive have Heartland's customers been to the cost of end-to-end encryption?

Carr: "We contracted with Voltage Security to use their encryption technology. We have absorbed that cost and the cost of developing an encryption advice. We are not passing that on to customers. We haven't increased anyone's pricing. That said, customers who want to go to our new encryption device will have to rent or buy it. It will cost under $500, approximately. The savings they'll get from not having card numbers in their systems will be worth it. The technology will prevent raw numbers from being transmitted in the clear."

Any pushback from customers on that one?

Carr: "We just rolled this out in late June and have numbers of merchants using it. Is there pushback? That remains to be determined. Many of our smaller customers can't even spell PCI. But the bigger customers are very receptive to this."

What's your single-biggest piece of advice for other companies that discover they've been the target of a data breach?

Carr: "What worked well is that when we announced it publically we had an all-hands meeting of all 3,000 employees. I told them their job was to be up front with our customers and tell them what it means for them. Let us be the one to tell them first, not the press. Being candid has been key. Some companies try to sweep it under the rug. Being pissed that this happened is important, too. I don't want this happening to anyone else, so we formed a payment-processing council to share information, share the malware samples, and help educate people, even our competitors. As I've gone around the country talking to people, there's a lot of chutzpa that this can't happen to them. The bad guys know all about the security methods employed in the industry. We need more humility. Those who feel comfortable with their security should ask themselves how they feel about their vulnerability to insider threats."

What should companies be asking in terms of the insider threat?

Carr: "Are there people inside their company who circumvent security policies in the name of being more efficient? Employees don't like what's inconvenient, and they find workarounds. How comfortable do you feel having data in the clear within your networks, where an insider can access it? How many IT security organizations have high-level management asking them to bypass certain security controls as a favor to them cause they are the boss? These are vital questions."