This joint report from Georgetown University’s Center for Security and Emerging Technology (CSET) and The Alan Turing Institute’s Centre for Emerging Technology and Security (CETaS) assesses the current state-of-the-art in autonomous cyber defence and its future potential, identifies barriers to progress and recommends specific action that can be taken to overcome those barriers.

Given the immense economic and societal damage caused by cyberattacks and recent advances in artificial intelligence (AI), interest in the application of AI to enhance cyber defence has grown in recent years. This report focuses on one promising approach to creating these autonomous cyber defence agents: reinforcement learning (RL).

While the breadth of promising and relevant modelling approaches, techniques and technologies that relate to autonomous cyber defence is large, our focus on RL is guided by the increased efforts in applied RL for cyber defence and the promising results RL has achieved in other problem domains. Academic publications have surged in recent years and gyms for training cyber-RL agents have begun to proliferate. However, capabilities remain rudimentary and incomplete compared to the more complex real-world network environments these agents will face. Sustained funding, coordinated effort to bolster simulation, emulation and evaluation tools, securing skilled personnel, and provisioning access to realistic data and infrastructure will help assure progress.

The research found that there is substantial potential for growth if technical challenges can be overcome. The existing agents and environments built for cyber defence currently consider fewer variables and possibilities than the more famous RL agents for playing board games like Go or video games like Atari or DOTA2. This means there is ample potential for increasingly intelligent agents; ones that can manage a larger number of possible defensive actions and operate in more complex environments that require them to explore more situations. Our exploration of the technical challenges revealed that autonomous cyber defence is going to be a long-term ambition that can only be realised years into the future.