Small language models (SLMs) have emerged as promising alternatives to large language models (LLMs) due to their low computational demands, enhanced privacy guarantees and comparable performance in specific domains through light-weight fine-tuning. Deploying SLMs on edge devices, such as smartphones and smart vehicles, has become a growing trend. However, the security implications of SLMs have received less attention than LLMs, particularly regarding jailbreak attacks, which is recognized as one of the top threats of LLMs by the OWASP. In this paper, we conduct the first large-scale empirical study of SLMs' vulnerabilities to jailbreak attacks. Through systematically evaluation on 63 SLMs from 15 mainstream SLM families against 8 state-of-the-art jailbreak methods, we demonstrate that 47.6% of evaluated SLMs show high susceptibility to jailbreak attacks (ASR > 40%) and 38.1% of them can not even resist direct harmful query (ASR > 50%). We further analyze the reasons behind the vulnerabilities and identify four key factors: model size, model architecture, training datasets and training techniques. Moreover, we assess the effectiveness of three prompt-level defense methods and find that none of them achieve perfect performance, with detection accuracy varying across different SLMs and attack methods. Notably, we point out that the inherent security awareness play a critical role in SLM security, and models with strong security awareness could timely terminate unsafe response with little reminder. Building upon the findings, we highlight the urgent need for security-by-design approaches in SLM development and provide valuable insights for building more trustworthy SLM ecosystem.

The widespread adoption of conversational LLMs for software development has raised new security concerns regarding the safety of LLM-generated content. Our motivational study outlines ChatGPT's potential in volunteering context-specific information to the developers, promoting safe coding practices. Motivated by this finding, we conduct a study to evaluate the degree of security awareness exhibited by three prominent LLMs: Claude 3, GPT-4, and Llama 3. We prompt these LLMs with Stack Overflow questions that contain vulnerable code to evaluate whether they merely provide answers to the questions or if they also warn users about the insecure code, thereby demonstrating a degree of security awareness. Further, we assess whether LLM responses provide information about the causes, exploits, and the potential fixes of the vulnerability, to help raise users' awareness. Our findings show that all three models struggle to accurately detect and warn users about vulnerabilities, achieving a detection rate of only 12.6% to 40% across our datasets. We also observe that the LLMs tend to identify certain types of vulnerabilities related to sensitive information exposure and improper input neutralization much more frequently than other types, such as those involving external control of file names or paths. Furthermore, when LLMs do issue security warnings, they often provide more information on the causes, exploits, and fixes of vulnerabilities compared to Stack Overflow responses. Finally, we provide an in-depth discussion on the implications of our findings and present a CLI-based prompting tool that can be used to generate significantly more secure LLM responses.

Lately it seems conversations about artificial intelligence (AI) are everywhere. There are constant discussions on the potential for popular AI chatbot ChatGPT, developed by OpenAI, to take over jobs ranging from media to analysts to the tech industry, and maybe even malicious phishing attacks. But can AI really replace humans? That's what recent research from Hoxhunt, a cybersecurity behavior change software company, hoped to explore by analyzing the effectiveness of ChatGPT-generated phishing attacks. The study analyzed more than 53,000 email users and compared the win-rate on simulated phishing attacks created by human social engineers and those created by AI large language models.

There are so many buzzwords and trends in the security awareness industry that it is hard to determine what is useful and what is a gimmick. Every vendor out there has some sort of promise that they have some special characteristic about their product that makes it a revolutionary improvement to your security awareness posture that no other product can accomplish. After reviewing the Hoxhunt solution, it is safe to say that they actually do provide something unique that can really move the needle with your organization's security awareness posture. Machine learning and artificial intelligence are typically buzzwords and technologies that vendors tout as making a product unique. The reality is that machine learning and AI can be useful, however, they are just underlying technologies.

Bias and susceptibility were evident during the 2016 US Presidential election and has plagued much of President Trump's first four years in office. The term "fake news," which years ago would have been considered absurd, is now part of our cultural vernacular. Allegations against foreign-state actors interfering with US elections and conspiracy theories related to COVID-19 has divided a culture, communities, friends, and even families. Social media has become a platform that propagates both real and fake news and has confounded the next generation of fact checkers and truth seekers dedicated to vetting accurate content. "Deep Fake" In recent years, the emergence of fake news has brought the concept deep fake to the public spotlight.