Tool-augmented large language models (LLMs) are rapidly being integrated into real-world applications. Due to the lack of benchmarks, the community still needs to fully understand the hallucination issues within these models. To address this challenge, we introduce a comprehensive diagnostic benchmark, ToolBH. Specifically, we assess the LLM's hallucinations through two perspectives: depth and breadth. In terms of depth, we propose a multi-level diagnostic process, including (1) solvability detection, (2) solution planning, and (3) missing-tool analysis. For breadth, we consider three scenarios based on the characteristics of the toolset: missing necessary tools, potential tools, and limited functionality tools. Furthermore, we developed seven tasks and collected 700 evaluation samples through multiple rounds of manual annotation. The results show the significant challenges presented by the ToolBH benchmark. The current advanced models Gemini-1.5-Pro and GPT-4o only achieve a total score of 45.3 and 37.0, respectively, on a scale of 100. In this benchmark, larger model parameters do not guarantee better performance; the training data and response strategies also play a crucial role in tool-enhanced LLM scenarios. Our diagnostic analysis indicates that the primary reason for model errors lies in assessing task solvability. Additionally, open-weight models suffer from performance drops with verbose replies, whereas proprietary models excel with longer reasoning.

Generative AI's expanding footprint across numerous industries has led to both excitement and increased scrutiny. This paper delves into the unique security challenges posed by Generative AI, and outlines potential research directions for managing these risks. Generative AI (GenAI) systems enable users to quickly generate high-quality content. GenAI models are designed to understand and generate content with a degree of autonomy that surpasses traditional machine learning systems, providing novel capabilities to generate text and code, interact with humans and Internet services, generate realistic images, and understand visual scenes. This capability enables a broader range of applications, and in this way introduces new security challenges unique to these novel GenAI-integrated applications. In this paper we discuss the challenges and opportunities for the field, starting in this section with the security risks, including how GenAI models might become a target of attack, a "fool" that unintentionally harms security, or a tool for bad actors to attack others. While GenAI models have groundbreaking capabilities, they are also susceptible to adversarial attack and manipulation. Jailbreaking and prompt injection are two prominent threats to GenAI models and applications built using them. Jailbreaking is an emergent technique where adversaries use specially crafted prompts to manipulate AI models into generating harmful or misleading outputs (Chao et al., 2023; Wei et al., 2023; Liu et al., 2023d). This exploitation can lead to the AI system bypassing its own safety protocols or ethical guidelines.

Command and control (C2) channels are an essential component of many types of cyber attacks, as they enable attackers to remotely control their malware-infected machines and execute harmful actions, such as propagating malicious code across networks, exfiltrating confidential data, or initiating distributed denial of service (DDoS) attacks. Identifying these C2 channels is therefore crucial in helping to mitigate and prevent cyber attacks. However, identifying C2 channels typically involves a manual process, requiring deep knowledge and expertise in cyber operations. In this paper, we propose a reinforcement learning (RL) based approach to automatically emulate C2 attack campaigns using both the normal (public) and the Tor networks. In addition, payload size and network firewalls are configured to simulate real-world attack scenarios. Results on a typical network configuration show that the RL agent can automatically discover resilient C2 attack paths utilizing both Tor-based and conventional communication channels, while also bypassing network firewalls.

Command and control (C2) paths for issuing commands to malware are sometimes the only indicators of its existence within networks. Identifying potential C2 channels is often a manually driven process that involves a deep understanding of cyber tradecraft. Efforts to improve discovery of these channels through using a reinforcement learning (RL) based approach that learns to automatically carry out C2 attack campaigns on large networks, where multiple defense layers are in place serves to drive efficiency for network operators. In this paper, we model C2 traffic flow as a three-stage process and formulate it as a Markov decision process (MDP) with the objective to maximize the number of valuable hosts whose data is exfiltrated. The approach also specifically models payload and defense mechanisms such as firewalls which is a novel contribution. The attack paths learned by the RL agent can in turn help the blue team identify high-priority vulnerabilities and develop improved defense strategies. The method is evaluated on a large network with more than a thousand hosts and the results demonstrate that the agent can effectively learn attack paths while avoiding firewalls.

Firewalls are critical components in securing communication networks by screening all incoming (and occasionally exiting) data packets. Filtering is carried out by comparing incoming data packets to a set of rules designed to prevent malicious code from entering the network. To regulate the flow of data packets entering and leaving a network, an Internet firewall keeps a track of all activity. While the primary function of log files is to aid in troubleshooting and diagnostics, the information they contain is also very relevant to system audits and forensics. Firewalls primary function is to prevent malicious data packets from being sent. In order to better defend against cyberattacks and understand when and how malicious actions are influencing the internet, it is necessary to examine log files. As a result, the firewall decides whether to 'allow,' 'deny,' 'drop,' or 'reset-both' the incoming and outgoing packets. In this research, we apply various categorization algorithms to make sense of data logged by a firewall device. Harmonic mean F1 score, recall, and sensitivity measurement data with a 99% accuracy score in the random forest technique are used to compare the classifier's performance. To be sure, the proposed characteristics did significantly contribute to enhancing the firewall classification rate, as seen by the high accuracy rates generated by the other methods.

As protests against rigid covid control measures in China engulfed social media in the past week, one Twitter account has emerged as the central source of information: @李老师不是你老师 ("Teacher Li Is Not Your Teacher"). People everywhere in China have sent protest footage and real-time updates to the account through private messages, and it has posted them, with the sender's identity hidden, on their behalf. The man behind the account, Li, is a Chinese painter based in Italy, who requested to be identified only by his last name in light of the security risks. He's been tirelessly posting footage around the clock to help people within China get information, and also to inform the wider world. The work has been taking its toll--he's received death threats, and police have visited his family back in China.

In today's digital world, it is more important than ever to ensure that your data is protected especially with the rise of machine learning also known as artificial intelligence (AI). Machine learning is a popular technology topic as it's becoming a part of our daily lives and can potentially have powerful implications for good and evil. In case you are not familiar with the terms machine learning or artificial intelligence, it is having the ability to train a computer to do something and learn over time so down the road it can infer what to do when faced with a basic task. Just a few examples of common consumer facing artificial intelligence machines are Apple's Siri, Google Assistant and Amazon's Alexa. With these machines learning our habits and likes/dislikes overtime, we are able to make our daily lives easier whether it's getting an answer to a question, directions to a local store or restaurant recommendations.

"64% of companies worldwide have experienced at least one form of a cyber-attack." "There were 22 billion breached records in 2021." "Every 39 seconds, there is a new attack somewhere on the web." Today, one of the biggest threats to organizations and businesses is cyber-attacks. Unfortunately, our go-to defense is firewalls.

Reinforcement learning (RL) has been applied to attack graphs for penetration testing, however, trained agents do not reflect reality because the attack graphs lack operational nuances typically captured within the intelligence preparation of the battlefield (IPB) that include notions of (cyber) terrain. In particular, current practice constructs attack graphs exclusively using the Common Vulnerability Scoring System (CVSS) and its components. We present methods for constructing attack graphs using notions from IPB on cyber terrain analysis of obstacles, avenues of approach, key terrain, observation and fields of fire, and cover and concealment. We demonstrate our methods on an example where firewalls are treated as obstacles and represented in (1) the reward space and (2) the state dynamics. We show that terrain analysis can be used to bring realism to attack graphs for RL.

A recent Forrester report states that organizations that apply artificial intelligence to cyber security will see a return on investment within 18 months; those who don't will wait three years. The report refers to artificial intelligence, or A.I., which includes machine learning, natural language processing, and analytics technology in cyber security efforts. Here are ten benefits of artificial intelligence in cyber security that every security professional should know about. A.I. is capable of handling large amounts of data and simplifies human resources in Cyber Security. With all these benefits, Artificial Intelligence simplifies life, making it secure by protecting it from cyber-attacks.