10 questions to answer before running a capture the flag (CTF) contest
- 06 March, 2018 22:00
What is CTF?
Capture the flag (CTF) contests are a way to teach people about real-world hacking and exploits in a fun environment. CTFs have been around for decades. One of the longest-running and more popular series began at the Vegas DEFCON show in 1996 and attracts thousands of participants. Since then, they have sprouted up everywhere and can be found in most cities of the world, as well as across numerous online contest websites. There are even CTFs designed for high school students.
Lately, many corporate IT departments are running their own events. These contests can take many forms, and you’ll need to answer several questions before you can build your playbook and conduct a successful event.
This short primer will help security teams to design their own CTF exercise. It suggests what types of challenges you need to include, how to make the contest run smoothly, and other logistics to consider. Let’s start by asking ten basic questions.
1. Will you use the CTF to recruit new employees?
There are many reasons for creating a contest, including general education of your management about cyber threats before your company experiences them first-hand, or using the contest as a team-building exercise. One of the more interesting objectives is to identify new cybersecurity talent either inside or outside of your organization. For example, this CTF run by the Air Force was used for its own recruiting.
Part of the problem is the number of unfilled infosec jobs. Greg Sparrow is the senior vice president of Compliance Point of Duluth, GA, a security VAR that has helped run a number of contests in the Atlanta area. “Demand for cybersecurity expertise usually outweighs their supply, so this can be a great way to identity people who have the skills to do this type of work.”
One reason they are useful is due to the inherent hands-on design of the average CTF. “You are limited to what you can learn in a real-world situation. Running CTFs helps with having more hands-on learning for recognizing and solving cyber threats. It isn’t enough to read a book about this that just talks about theory. A CTF is the closest you can get to the real world without running afoul of the legal system,” says Sparrow.
One IT manager told me that, “We look at our CTF events as security awareness initiatives for our internal employees. Our two-day events are split into two parts: a morning educational session where we spend time educating the attendees on hacking techniques, and an afternoon session where they compete in small groups against each other.” They found their events useful because “each attendee comes out of it in a way that strengthens our overall security posture.”
2. What age group and experience level will you aim for?
CTFs can be run for all ages, even for high school students as mentioned earlier. So, when you design your contest, consider the audience and whom you want as ideal participants. This is true even for an in-house event comprised of your own staff. If you want to attract people from around your company, you may want to put out a couple of pre-competition qualifying questions to set expectations — and screen potential participants — accordingly.
3. Should you use a commercial cyber range operator or set up a CTF?
Cyber simulations or ranges are dedicated places that are set up to conduct CTF-type exercises that are run by security consultants. This may be an alternative to running your own contest, since they have created pre-built scenarios to help illustrate common security threats.
They are popping up at many places around the world: Baltimore has one run by Cyberbit for example. I wrote about going to one of them in Israel called CyberGym two years ago, and CSO has written about the one in Michigan. It might make sense to try out a cyber range initially, or to attend a local CTF event, to get your feet wet and understand how things are constructed as well as to clarify your own learning objectives if you decide to run your own contest later on.
4. What type of CTF contest do you want to run?
Generally, there are two types of contests: Jeopardy and red/blue challenges. The first is pretty self-explanatory, with a collection of questions similar to the TV quiz show and arranged in different categories.
The second is the more classic format, which is what most people think of. This involves two groups, one defending the network (the blue team) and one of attackers (the red team). The two sides usually switch half-way through the contest, so everyone gets a chance to try from both perspectives.
You can also run a contest that is a mixture of the two types. This academic paper goes into more details about other CTF varieties. “There is a lot of value to be gained from doing both sets of exercises,” says Sparrow.
5. What categories of challenge questions will you use?
Most CTFs mix up their challenge questions into several categories, such as steganography, cryptography, mobile OS exploits, app-specific exploits (web, email, file sharing), reverse engineering, forensics, programming, and penetration testing challenges. Depending on what you want to accomplish, you may want to focus on only a couple of categories.
6. What prizes, entry fees, and time length will you use?
As you design your contest, consider the logistical factors and make sure you are clear about communicating what are the prizes and how long the contest will actually run. Everyone likes to get a prize of some sort to motivate them, and CTFs are no exception.
7. Will the CTF be open to the public?
Some CTFs are actual public events where you have to travel to a physical location and sit in a room with the other contestants. Some are strictly online only (see our resource list) where all you need is a web browser and a set of hacking tools. Some are closed for corporate-only or invitation-only events. Again, pick what is most appropriate for your needs.
8. Where will you get your challenge questions (and any hints)?
Figuring this out in advance is usually one of the more time-consuming tasks. Guanyu Tian is an assistant professor of computer science at Fontbonne University in St. Louis. He has hosted high-school level CTFs to recruit students to apply to his undergraduate program. He says, “You have to consider the problem set difficulty for the participants.”
For example, for his teenage competitors he was surprised that they were able to blow through most of his problems in a couple of hours. “We ended up needing additional challenges to keep the competition going. Our faculty has had lots of previous experience in participating in other competitions. So even though we knew the type of questions to pose, it still took some effort to figure out the right match of problems to participants,” says Tian.
Tony Bryan, the executive director of the Midwest Cyber Center, who has run several CTFs for teens and adults, agrees with this sentiment. “It is important to figure out the appropriate level of knowledge and interest ahead of time. Also, you should provide hints for people when they get stuck on a challenge question too.”
Our resource list has sources for sample challenge questions to help get you started.
9. Will you run the contest on an external infrastructure or your own?
The one aspect many new contest creators forget about is how to score the contestants and keep track of each of the teams’ progress in solving the various challenges. Typically, this is displayed on a monitor for the participants to view as they complete each challenge. You can DIY, hire a consultant, or use a variety of open source solutions to create the scoreboard display. (Our resource list has several links to these tools.)
Sparrow used homegrown solutions for data capture and reporting components. A company’s IT manager told me that, “Our first event was hosted by a third party, on our premises. They provided the server, scoring system and course materials; we provided the network infrastructure and laptops. We used this third party the first time to help us figure out what exactly we wanted and for ease of turnaround.” After that event, they were able to run their own CTFs internally and built their own scoring systems.
10. How will you staff your contest?
You will need all sorts of people to run your contest, and this is where attending one of the local CTFs will help you understand the lay of the land and the different jobs to be filled. Obviously, you need judges and people to set up the equipment and the room that you will be using.
For example, one company uses a mix of their own IT security architects and penetration testers to act as both coaches and event facilitators. Sparrow gets his judges from several Atlanta meetups and IT organizations, and that could be a useful source for your own contest.
Resource list
Challenge question sources
Online CTF contests
An online CTF happens almost every day of the year somewhere in the world. Some of these websites require you to be part of a team, others allow individuals, and some charge a small fee to participate.
Hints, hacking tools, and helpful tips for participants
- Slothkrew CTF Playbook
- Trail of Bits CTF Field Guide
- Apsdehal Awesome CTF
- The College of Computing at Georgia Tech CTF and Pentest Training Tools
