CINTEL for Cybersecurity

Current approaches to AI integration in Security Operations Centres (SOCs) rely heavily on automation and augmentation, approaches with limitations hindering their effectiveness.  We investigate the role of human-AI teaming, fostering active collaboration that unlocks the true synergy between human expertise and AI capabilities.

The Challenge

Working in cybersecurity can be a stressful job for human workers as automated monitoring systems generate large numbers of alerts which require attention. Distinguishing and prioritising the most significant threats from large volumes of warnings can be an overwhelming task, and, in practice, human analysts often ignore much of what is presented to them by the artificially intelligent cybersecurity systems they work with. Even when humans use automated systems, human knowledge and intelligence is still required given the constantly changing nature of cybersecurity threats. There is, therefore, a need to better design these systems to improve the collaboration between human experts and artificially intelligent algorithms in order to identify novel threats and better prioritise responses to the various alerts that are constantly being generated.

Our Response

This project looks at how to make cybersecurity operations more effective by leveraging the strengths of both human security experts and AI systems. Instead of taking a human-in-the-loop approach to decision-making, it focusses on AI-in-the-loop to augment and improve human performance.

Impact

Cybersecurity is a vital issue for governments, organisations and individuals. So finding better ways to combine human and AI expertise will improve our ability to respond effectively to new and existing threats. The type of human-AI collaborative surveillance systems developed in this project can also inform many other domains which face similar problems, with human operators dealing with alerts from automated systems. Examples include the astronomy anomaly detection https://research.csiro.au/cintel/projects/collaborative-data-cleaning-and-anomaly-detection-in-complex-control-systems/ project within the CINTEL Future Science Platform.

Team

Mohan Baruwal Chhetri, Fatemeh Jalalvand, Shahroz Tariq, Ronal Singh, Surya Nepal, Cécile Paris

Students

Karunasingha Gedara Navodika Madushan Karunasingha is a CINTEL-funded PhD student at UNSW, with Professor Salil Kanhere, Mohan Baruwal Chhetri, Surya Nepal and Cécile Paris. Navodika has recently successfully completed his PhD confirmation review. His PhD topic is broadly in the area of Collaborative Surveillance in Security Operations Centres. Navodika is specifically looking at the role of situation awareness under the different modes of decision-making in the  𝒜2 𝒞 Framework, namely Automation, Augmentation, and Collaboration.

Publications

  • Shahroz Tariq, Mohan Baruwal Chhetri, Surya Nepal, Cécile Paris (2025). Alert Fatigue in Security Operations Centres: Research Challenges and Opportunities. Accepted for publication in ACM Computing Surveys on 5 March 2025.
    • This paper reviews existing literature and industry solutions for mitigating alert fatigue in Security Operations Centres (SOCs) through automation, augmentation, and human-AI collaboration. It identifies key causes of alert fatigue, examines the limitations of current solutions, and proposes future research directions leveraging AI to enhance SOC efficiency and reduce analyst burnout.
  • Hoang Cuong Nguyen, Shahroz Tariq, Mohan Baruwal Chhetri, Quoc Bao Vo (2025). Towards Accurate CTI Extraction: A Summarisation and Classification Approach Using Large Language Models. To be published at the International World Wide Web Conference 2025 ( WWW2025 | The Web Conf 2025 ) 28 April – 2 May, 2025, Sydney, Australia.

    • This paper evaluates the performance of four state-of-the-art Cyber Threat Intelligence extraction methods using the MITRE ATT&CK framework, identifying key challenges such as class imbalance and overfitting. To overcome these issues, it proposes a two-step pipeline combining LLM summarisation with a retrained SciBERT model, resulting in improved performance over the baseline models.

  • Fatemeh Jalalvand, Mohan Baruwal Chhetri, Surya Nepal, Cécile Paris (2024): Alert Prioritisation in Security Operations Centres: A Systematic Survey on Criteria and Methods. ACM Computing Surveys. Vol. 57, No. 2, Article 42.
    Publication date: November 2024. doi.org/10.1145/3695462, (IF: 23.8, ranked 1/143 Journals in Computer Science Theory & Methods).

    • This paper provides a comprehensive review of the criteria and methods used for alert prioritisation in Security Operations Centres. It analyses their advantages and disadvantages using the lens of human-AI teaming, specifically automation, augmentation, and collaboration.

  • Mohan Baruwal Chhetri, Shahroz Tariq, Ronal Singh, Fatemeh Jalalvand, Cécile Paris, Surya Nepal (2024). Towards Human-AI Teaming to Mitigate Alert Fatigue in Security Operations Centres. ACM Transactions on Internet Technology 24 (3), 1-22.
    • This vision paper proposes the 𝒜2 𝒞 Framework, which enables flexible and dynamic decision making in human-AI teams by allowing seamless transitions between automated, augmented, and collaborative modes of operation. It allows AI-powered automation for routine alerts, AI-driven augmentation for expedited expert decision making, and collaborative exploration for tackling complex, novel threats.