Paper: CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios

Authors: Taein Lim (Chung-Ang University, undergraduate), Seongyong Ju (Chung-Ang University, undergraduate), Munhyeok Kim (Chung-Ang University, undergraduate), Hyunjun Kim (Myongji University), Hoki Kim (Chung-Ang University)

Venue: NeurIPS 2026

URL: https://arxiv.org/abs/2605.07830

Original Edaily article

Undergraduate Researchers' Paper from the Department of Industrial Security Accepted at a Leading AI Conference
Selection rate (solid) and attack success rate (dashed) per attack type for five AI agents. Each agent clearly prefers different attack types.

A paper by Chung-Ang University undergraduates will be presented at an international conference.

Chung-Ang University announced on the 19th that a paper first-authored by undergraduates from the Department of Industrial Security has been accepted for presentation at NeurIPS 2026, a leading international conference in artificial intelligence. Taein Lim, Seongyong Ju, and Munhyeok Kim are the first authors of the paper on AI security agents (CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios), advised by Prof. Hyunjun Kim (Myongji University) and Prof. Hoki Kim (Chung-Ang University).

AI agents have advanced to the point of calling tools, planning attacks, and carrying out penetration testing and vulnerability discovery on their own. Yet most AI-security evaluations have focused on whether an attack succeeded, that is, the attack success rate. Which attacks an agent actually chooses and repeats, even among agents with the same success rate, has not been adequately evaluated.

To address this, the research team defined the tendency of AI agents to favor particular types of cyber attack as “attack-selection bias” and proposed a benchmark to measure it. The team also analyzed which attacks the agents choose together with whether those attacks actually succeed. The experiments showed that each AI agent clearly prefers different attack types.

Prof. Hoki Kim said, “By exploiting each agent’s attack bias, red teaming can combine multiple agents that complement one another and explore a wider range of vulnerabilities,” adding, “We plan to actively apply the results to the Cybersecurity-Specialized AI Foundation Model project we are currently participating in.”

The research was supported by the National Research Foundation of Korea, among others.