Capturing the Flag with ChatGPT: Generative AI for Cyber Education

Since the release of ChatGPT in November 2022, generative AI chatbots have captured the attention of practically everyone in education. Students are eager to adopt AI tools to help them with assignments, while teachers and administrators are scrambling to adapt to the presence of these tools in the classroom. In certain cases, however, these tools can make a real difference in students’ ability to learn complex or technical concepts. This is especially important in cybersecurity, where the United States faces a persistent shortage of talent and many students are not exposed to cyber-relevant opportunities until college.

Key Takeaways

ChatGPT is an effective learning tool for students preparing for CTF competitions, but it has some important limitations.

ChatGPT is most helpful at summarizing and synthesizing information, especially for beginners. It struggles more with ciphers and coding, suggesting that it might be less helpful for more difficult cyber problems.

New students may want to consider adding ChatGPT to their toolkits when learning cyber concepts and preparing for their first competitions. However, there are still many things we don’t know about how ChatGPT will impact cyber education.

Introduction

The combination of cyber competitions and ChatGPT holds the potential to elevate cyber education, teaching students how to crack passwords, break websites, and dive deep into the internet. ChatGPT is surprisingly good at explaining tough cyber concepts that many beginner learners struggle with. While it has its flaws, it shouldn’t be overlooked as a teaching and learning tool that can help get students interested in learning cyber skills and pursuing cyber careers.

ChatGPT provides one way students can develop new cybersecurity skills, namely by helping them prepare for their first Capture the Flag competitions, also known as CTFs. These competitions are a popular entry point into cybersecurity for students of all ages and experience levels, and multiple publications have previously advocated for the significant value of CTFs to cyber education.

What is a CTF?

CTFs expose middle school, high school, and college students to a fun and challenging way to learn about cybersecurity concepts like network security, cryptography, and web security. These competitions are designed to allow students to safely practice skills and gain real-world experiences in identifying and exploiting vulnerabilities in a controlled environment. Not only do CTFs prepare students with knowledge, they also help foster team collaboration, which develops students’ communication and problem-solving skills. Overall, participating in CTFs is a valuable way for students to prepare for a career in cybersecurity and stay ahead of the ever-evolving threat landscape.

Given the dynamic nature of the tech industry, fostering the next generation of cyber talent is more important now than ever. While sparking students’ initial interest in cybersecurity is crucial, it’s equally important to provide ongoing support and resources to help them develop their skills. Generative AI tools can come into play, an empowering resource to aid in growing students’ talent.

Case Study: Cyber Fasttrack

With the help of CSET researchers Jessica Ji and Jenny Jun, I was able to design and execute a case study analyzing the effectiveness of using ChatGPT on a particular CTF, Cyber Fasttrack. I wanted to compare it to my previous experience of participating in CTFs without generative AI tools like ChatGPT, with hopes of finding out how ChatGPT could be used for CTFs and cyber education in general.

Here are the research questions we framed our case study around:

How can generative AI tools such as ChatGPT improve the CTF experience before, during, and after participation?

Where do AI tools like ChatGPT and code generation tools fall short? In what way?

One of the challenges in cyber education is ensuring that students have access to digestible learning content to retain their interest. In preparation for a CTF, students can use generative AI tools to explain cybersecurity concepts and tools geared to their experience level––whether it’s beginner, intermediate, or advanced. A valuable software tool commonly used in the network security category in CTFs is Wireshark. If a beginner Googled “What is Wireshark and How is it Used?” an article from CompTIA would pop up––clicking to read more, students can get overwhelmed by how much they would have to scroll and digest. Instead, for someone who is just starting to gain interest in cybersecurity concepts, ChatGPT presents information in a less daunting way. Here is the comparison:

Look at all that scrolling…

During actual CTF competitions, ChatGPT also is capable of assisting students as a tool to solve certain ciphers, learn different command lines used in the Linux terminal, and explain how to code a certain prompt. Using questions from my Cyber Fasttrack experience, here are the conversations I had with ChatGPT given the type of questions:

Cryptography: Solving Ciphers

Rather than a student manually lining up the alphabet with its index, ChatGPT explains the cipher a little bit and also helps answer the challenge. However, it is important to note that ChatGPT does have shortcomings, even for some easy ciphers. When prompted with a Caesar Cipher question, ChatGPT was unable to produce the correct answer after multiple tries of rephrasing the question and guiding it. However, it was able to efficiently explain what a Caesar Cipher is, how to apply it, and to work through an example of one.

After a CTF is over, ChatGPT can also be used as a study tool to retain and practice cybersecurity skills or concepts from the competition, as users can prompt the tool with mock quizzes and questions.

Throughout the case study, I discovered, perhaps not surprisingly, that ChatGPT struggled more with questions as the difficulty increased. Similarly, for code generation (see the “Further Information” section for an example using the Tower of Hanoi problem), it might not output the code you’re looking for on its first try, but with some coaching and guidance, it can definitely get there.

Analysis

Though it was surprising to see that ChatGPT couldn’t solve a relatively common Caesar Cipher, it still does well in explaining the concept. Most likely due to its tendency to highlight patterns and recognize repeated text, it’s probably a better resource for definitions and summaries than for cracking ciphers. Ciphers are designed to obfuscate messages through hiding these patterns and keywords, making them harder for AI tools to decipher. However, as its capabilities improve, ChatGPT may eventually be able to recognize and decode ciphers.

Currently, ChatGPT essentially serves as a more user-friendly search engine, though it is still a powerful tool that can help students and CTF participants find and refine information in a more digestible way. With its advanced natural language processing capabilities, ChatGPT can help to quickly filter through complex information and provide relevant and concise content that may be more difficult to obtain through traditional search engines. This makes it an invaluable resource for anyone looking to gain an edge in their cybersecurity studies, especially for beginners who may be struggling to understand new tools and concepts.

It is important to note, like everything else on the internet, information is not always guaranteed to be accurate, which is something to keep in mind while using ChatGPT and non-AI tools alike. They can lead you astray from the answer, output incorrect information, and just use up your time––which can be costly in a time-sensitive CTF.

Conclusion and Key Takeaways

As technology continues to evolve, developing future cyber talent remains a necessity. CTF competitions are one way to spark students’ passion for cybersecurity. As shown in this case study, generative AI tools such as ChatGPT show early promise in helping students prepare for beginner-to-intermediate-level CTFs and succeed in cybersecurity.

ChatGPT can be a beneficial tool for CTF participants partly because it relays relevant information in a more digestible format. If I had access to ChatGPT as I was initially learning cybersecurity in high school, I believe it would have helped me reinforce and retain cyber concepts. However, like all technology, there are flaws to be mindful of. Just as the internet is full of unreliable data, ChatGPT may output inaccurate information and can lead students astray and down unhelpful rabbit holes. As it continues to improve over time, it does have the potential to become a significant resource for CTF participants.

With the limited time to perform this case study, there were some questions that arose for future research:

Is there a generative AI tool that CTF participants prefer?
- As more tools are released to the public, there’s an opportunity to compare ChatGPT, Microsoft Bing, and Google’s Bard.

Are there differences between beginner-friendly to professional-level CTFs, and whether or not one AI tool is preferable over another?
- For example, is ChatGPT preferred for beginner-friendly CTFs and Google’s Bard preferred for more intense CTFs?

In the future, I hope to look further into exploring the questions mentioned above, along with also researching the flip side of how ChatGPT can be used as a weapon for cyber actors. As tech consistently develops, I am eager to see the growth in tech and cyber.

— —

Lisa Lam is a former CSET undergraduate intern majoring in computer science at Saint Joseph’s University. She interned with the CyberAI Project during the spring 2023 semester as part of the Washington Center’s Cybersecurity Accelerator Program. She has capture-the-flag competition experience at both the high school and college levels and leveraged her experience to conduct an independent research project on the utility of generative AI for cyber education.