10 tips for effective threat hunting

By Kane Lightowler, Carbon Black

Attackers don’t think of their success as optional, hence the effectiveness and success of a threat hunting program is critical. Organisations that start such a program have success in mind, but are they able to achieve it? The ten tips below help organisations and their threat hunters be effective and successful.

1 – Know your environment

Threat hunting is focused on the discovery of abnormal activities that point directly to reconnaissance and attacks. To recognise activities that aren’t normal, it’s important to understand what’s normal. Further, it’s important to become familiar with the architecture overall and at a detailed level to understand where vulnerabilities and weaknesses are that attackers could target.

Understanding one’s environment involves deep and wide exploration of the technical environment: networks, systems, and applications. But it’s also imperative that a threat hunter should also build relationships with key personnel in and outside of IT.

Why build relationships? These people help threat hunters to better understand normal activity versus anomalous activity. When a threat hunter finds a problem, it’s not always an attacker, but sometimes it’s an unsafe practice. Without a trusting relationship between threat hunters and others, the hunters cannot be effective agents of change to help the organisation make key security improvements and keep its house in order. 

2 – Think like an attacker A threat hunter’s mission is to find signs of intrusion, and quickly, so attacks can be stopped and their effects mitigated to minimise damage. But rather than adopting the mindset of always chasing attackers, better threat hunters anticipate their next move.

In a threat hunt, this process involves looking for things that attackers might do. With tools like endpoint security solutions, threat hunters can set up triggers that fire when an attacker does those things. This practice is also known as laying tripwires, which are triggers that a threat hunter sets up, anticipating an attacker’s move, and alerting personnel if such a move is made.

3 – Develop the OODA mindset

Observe. Orient. Decide. Act. This is how the military thinks about combat operations. Threat hunters are soldiers in the cyberwars, so it makes sense to think about threat hunting in this way.

OODA is mental discipline that keeps threat hunters from acting impulsively. In the cyber war arena, acting before thinking can blunt a threat hunter’s effectiveness.

4 – Devote sufficient resources to the hunt Threat hunting can be a great idea that goes sour if there aren’t enough resources to carry it out properly. This includes both personnel as well as tools and systems. Further, it includes personnel who know how to carry out threat hunts. Here’s a breakdown on what’s needed:

✓ Personnel: One or more trained and/or experienced threat hunters. These have a deep understanding on the inner workings of operating systems, plus sub-systems such as web servers, database management systems, and application servers. They need to have a thorough and growing familiarity of the inner workings of the organisation, as well as its applications, networks and users.

✓ Tools: A threat hunt cannot start without threat hunting tools. This includes an endpoi9nt security solution installed on every endpoint, which provides a step‐by-step detailed forensic history of every activity on every endpoint. The real power of such a tool is its central querying capability that allows a threat hunter to create and store queries, asking about whether certain detailed events have occurred anywhere in the environment.

✓ Infrastructure: Threat hunting requires some systems resources. These include management consoles, and possibly a ‘test range’ where advanced threat hunters can experiment with suspected malware in a safe environment. Here, hunters can hone their skills with ‘live fire’ and enhance their hunting skills in production environments.

5 – Deploy endpoint intel across the enterprise

In cyber warfare defenders must protect all endpoints all the time, but attackers only need to be successful once. This principle underscores the urgent need for an organisation to cover not just a subset of endpoints with advanced threat hunting tools, but all endpoints.

Leaving some endpoints unguarded creates blind spots where organisations are unable to detect or remediate attacks.

6 – Supplement endpoint intel with network intel

Endpoints are the hills on the cyber warfare battleground. While they’re the principal focus of attacks by intruders, endpoints are by no means the only place where information on intruders can be found. It is often useful to have network‐centric visibility by using tools, such as:

✓ Intrusion detection systems (IDS) ✓ Intrusion prevention systems (IPS) ✓ Netflow ✓ Web filters ✓ Firewalls ✓ Data loss prevention (DLP) systems

These tools provide a network‐centric view of activities that may help a threat hunter corroborate attack patterns and activities. Collecting additional intel from the network and other sources is a part of Observe and Orient.

7 – Collaborate across IT Threat hunting isn’t just about technology. The essential ingredient is strategic relationships with key personnel in the IT organisation. Better threat hunters work with systems engineers, network engineers, endpoint engineers, service desks and application developers in different ways:

✓ Understanding normal: As threat hunters build their knowledge of environments, they’ll be in dialogue with key IT personnel to hone their understanding on how systems and applications function.

✓ Remediation of vulnerabilities: While searching for intruders, threat hunters also encounter weaknesses in the design and implementation of applications, systems and networks. Relationships built on trust enable threat hunters to convey the need to fix those weaknesses.

✓ Remediation of incidents: When threat hunters find signs of intrusion, they need to work with key IT personnel to correctly diagnose intrusions and remediate them effectively and completely with minimal impact.

The OODA methodology applies perfectly here. Using their relationships across IT, they collect information (Observe), and work with others to understand it (Orient), before acting on it (Decide and Action). With relationships based on trust, IT personnel are more likely to cooperate with threat hunters to reduce risks in the organisation. 

8 – Keep track of those hunts

Even a single threat hunt can have more details than most people can remember. But over time, when a single threat hunter has performed 10, 20, 30 or more threat hunts, the details quickly become a blur.

For this reason, threat hunters should document each hunt. Better operators include important high level business information with each hunt, most notably the reason for the hunt.

A detailed history of threat hunts helps a threat hunter to better understand, at any level of detail, the ground that’s been covered, what’s been looked at, and what’s been overlooked. While it’s important to revisit old hunts sometime (meaning repeating a prior threat hunt if the threat hunter suspects intrusions since last time), IT environments quickly change over time, potentially leading to new intrusions by using methods examined earlier.

9 – Hone security skills

Innovation in the cyber security arms race is occurring at a dizzying pace. Seasoned threat hunters know this and take time out from the hunt to hone their skills through:

✓ Technical training: The SANS Institute (www.sans.org) and other organisations provide high‐quality technical training in attack and defence techniques.

✓ Conferences: Local gatherings, as well as national and international conferences like RSA, Black Hat, and DEFCON, provide tremendous networking and education opportunities.

10 – Be aware of attack trends

Threat hunters can’t exist on intellectual islands. Instead, they need to be continually aware of the techniques used by cyber criminal organisations against other organisations. Only with this knowledge can a threat hunter anticipate attacks and be able to find them. 

Tags IT SecuritySANS InstituteCarbon Blackthreat huntingOODA mindset

Show Comments