AI Security
AI Security Trends 2025: Market Overview & Statistics
Explore the latest AI security trends as businesses race to leverage AI, balancing its benefits with emerging threats and challenges.
Learn how to protect against the most common LLM vulnerabilities
Download this guide to delve into the most common LLM security risks and ways to mitigate them.
In-context learning
As users increasingly rely on Large Language Models (LLMs) to accomplish their daily tasks, their concerns about the potential leakage of private data by these models have surged.
[Provide the input text here]
[Provide the input text here]
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
Lorem ipsum dolor sit amet, Q: I had 10 cookies. I ate 2 of them, and then I gave 5 of them to my friend. My grandma gave me another 2boxes of cookies, with 2 cookies inside each box. How many cookies do I have now?
Title italic
A: At the beginning there was 10 cookies, then 2 of them were eaten, so 8 cookies were left. Then 5 cookieswere given toa friend, so 3 cookies were left. 3 cookies + 2 boxes of 2 cookies (4 cookies) = 7 cookies. Youhave 7 cookies.
English to French Translation:
Q: A bartender had 20 pints. One customer has broken one pint, another has broken 5 pints. A bartender boughtthree boxes, 4 pints in each. How many pints does bartender have now?
Lorem ipsum dolor sit amet, line first
line second
line third
Lorem ipsum dolor sit amet, Q: I had 10 cookies. I ate 2 of them, and then I gave 5 of them to my friend. My grandma gave me another 2boxes of cookies, with 2 cookies inside each box. How many cookies do I have now?
Title italic Title italicTitle italicTitle italicTitle italicTitle italicTitle italic
A: At the beginning there was 10 cookies, then 2 of them were eaten, so 8 cookies were left. Then 5 cookieswere given toa friend, so 3 cookies were left. 3 cookies + 2 boxes of 2 cookies (4 cookies) = 7 cookies. Youhave 7 cookies.
English to French Translation:
Q: A bartender had 20 pints. One customer has broken one pint, another has broken 5 pints. A bartender boughtthree boxes, 4 pints in each. How many pints does bartender have now?
Currently, 49% of firms use tools like ChatGPT across departments, from IT and marketing to finance and customer service (masterofcode).
However, as businesses rely more on AI, they must carefully manage the security risks that come with it. Compliance, privacy, and ethics have always been essential for companies, but these challenges have compounded with generative AI.
AI technology may be paradoxical due to its binary security considerations. For instance, 93% of security professionals say that AI can ensure cybersecurity, but at the same time, 77% of organizations find themselves unprepared to defend against AI threats (Wifitalents).
According to Immuta’s 2024 State of Data Security Report, 80% of data experts agree that AI is making data security more challenging.
So, how are companies incorporating AI security into their operations? What are the security concerns, and how businesses are preparing to overcome them?
Let’s see the latest AI security trends to find answers to these questions!
Artificial intelligence (AI) has become a core part of business operations.
Latest AI Security Trends
As AI adoption accelerates across industries, so do the threats against businesses, disrupting conventional security posture and readiness. Some driving factors include generative AI, AI-powered malware, evolving regulations, etc. Here, we will discuss the rapid shifts in AI security, from the searching market valuations to the emerging security concerns over adoption and regulation.
1. Market Overview
The stats mentioned below illustrate the overall size of AI in cybersecurity. The market is growing rapidly due to the high velocity, increased sophistication, sheer volume of cyber threats, and the benefits of AI-powered solutions.
- 90% of organizations are actively implementing or planning to explore large language model (LLM) use cases, while only 5% feel highly confident in their AI security preparedness. (Lakera)
- The global AI in cybersecurity market size was valued at $22.4 billion in 2023 and is expected to grow at a CAGR of 21.9% from 2023 to 2028. (MarketsandMarkets)
- The AI cybersecurity market is forecasted to double by 2026 before reaching 134 billion U.S. dollars by 2030. (Statista)
- AI/ML tool usage skyrocketed by 594.82%, rising from 521 million AI/ML-driven transactions in April 2023 to 3.1 billion monthly by January 2024. (Zscaler 2024 AI Security Report)
- 82% of IT decision-makers planned to invest in AI-driven cybersecurity in the next two years, and 48% planned to invest before the end of 2023. (Blackberry)
- There are 2,826 Artificial Intelligence (AI) companies in Cybersecurity worldwide. The leading companies include Splunk, Palo Alto Networks, Darktrace, CrowdStrike, Ping Identity, and Fortinet.
- Generative AI investment skyrockets. Despite a decline in overall AI private investment last year, funding for generative AI surged, reaching $25.2 billion. Major players in the generative AI space, including OpenAI, Anthropic, Hugging Face, and Inflection, reported substantial fundraising rounds. (AI Index Report 2024)
2. Security Concerns
Even as investments in AI explode, it is widely accepted that defensive AI will enhance cyber security. While there are reasonable grounds to believe AI can improve the security of cyber systems, there's also a clear understanding of the risks and challenges it brings.
When asked about the biggest concerns regarding AI in security in a survey, respondents expressed the following concerns: (CSA)
Confidence levels in current security measures are low, with only 5% of respondents rating their confidence at five out of five.
There is uncertainty about the effectiveness of existing security approaches in protecting against sophisticated AI attacks, with 86% having moderate or low confidence levels. (Lakera)
- Nearly 50% of respondents to a 2023 survey among global business and cyber leaders highlighted the advance of adversarial capabilities, such as phishing, malware development, and deep fakes, as their greatest concern regarding the impact of generative artificial intelligence (AI) on cybersecurity. (Statista)
- 60% of respondents fear their organizations are inadequately prepared to defend against AI-powered attacks. (Darktrace)
- Enterprises are blocking 18.5% of all AI and machine learning (ML) transactions—a 577% increase in blocked transactions over nine months—reflecting growing concerns around AI data security and companies’ reluctance to establish AI policies. (Zscaler)
- 77% of companies experienced breaches in their AI systems over the past year. (HiddenLayer)
- More than 95% of respondents believe dynamic content through LLMs makes detecting phishing attempts more challenging. (LastPass)
- 63% of organizations have established limitations on what data can be entered into GenAI tools, and 27% have banned GenAI applications altogether. (Cisco)
- 55% of data leaders say inadvertent exposure of sensitive information by LLMs is one of the biggest threats. (Immuta)
- 57% say they’ve seen a significant increase in AI-powered attacks in the past year. (Immuta)
- AI-powered attacks were the number one concern of 36% of respondents in a survey. (Splunk)
- 51% of IT decision-makers believe a successful cyberattack will be credited to ChatGPT within the year. (Blackberry)
3. Adoption and Regulation
Organizations are at various stages of GenAI, in particular, LLM adoption. 42% actively use and implement LLMs across various functions, indicating a significant commitment to leveraging AI capabilities for business innovation and growth.
Another 45% are exploring use cases and integration possibilities, reflecting a high level of interest and recognition of GenAI’s potential value.
Only 9% have yet to make plans to adopt LLMs, suggesting strong industry-wide momentum towards AI adoption and highlighting the competitive risk for organizations not embracing these technologies. (Lakera)
- The European Union (EU) is leading the way in establishing comprehensive regulations for AI, with the EU AI Act highlighting its commitment to safe and ethical AI development.
- 95% of IT decision-makers believe governments are responsible for regulating advanced technologies, such as ChatGPT. (Blackberry)
- ChatGPT usage continues to surge, with 634.1% growth, even though it is also the most-blocked AI application by enterprises. (Zscaler)
- 85% say they feel confident that their data security strategy will keep pace with the evolution of AI. (Immuta)
- 35% of companies reported using AI in their business for different purposes. (PWC)
- Younger generations tend to adopt AI technology in their professional life easily and quickly. (PWC)
- 2024 is set to be a revolutionary year for AI implementation in the security sector. 55% of organizations are planning to implement gen AI solutions. (CSA)
- The number of AI-related regulations in the U.S. has risen significantly in the past year and over the last five years. In 2023, there were 25 AI-related regulations, up from just one in 2016. Last year alone, the total number of AI-related regulations grew by 56.3%. (AI Index Report 2024)
- In 2023, 55% of organizations used AI (including generative AI) in at least one business unit or function, up from 50% in 2022 and 20% in 2017. (McKinsey)
- 36% of respondents said they hadn’t used AI and ML for cybersecurity but are currently “seriously exploring” generative AI tools. (CompTIA)
- 37% of data leaders say they have a comprehensive strategy in place to remain compliant with recent and forthcoming AI regulations and data security needs. (Immuta)
- 71% of organizations have already taken steps to minimize risks associated with adopting AI. (DarkTrace)
- 65% of respondents report that their organizations regularly use gen AI, nearly double the percentage from our previous survey ten months ago. (McKinsey)
4. Technological Developments
With fast-paced advancements in AI technology, its integration into the existing workflows will improve operational efficiency and reduce costs. Moreover, AI is viewed as a tool that complements the work of security professionals, not as something that will replace them.
In a survey, most professionals admitted they face challenges in threat investigation and response, with only 12% saying they have no difficulties in this area. This highlights AI's role in empowering professionals in tasks such as threat detection, incident response, etc., rather than replacing them. (CSA)
Only 12% of security professionals in that survey think AI will fully take over their jobs. At the same time, the majority believe it will either improve their skills, support their role, or take over significant parts of their tasks, allowing them to focus on other responsibilities. (CSA)
- Organizations are exploring a diverse range of use cases for AI systems, with the top use cases being rule creation (21%), attack simulation (19%), and compliance violation detection (19%). (CSA)
- Data leaders are also interested in the potential for AI as a data security tool. Respondents say that some of the main advantages of AI for data security operations will include: (Immuta)
- Anomaly detection (14%)
- Security app development (14%)
- Phishing attack identification (13%)
- Security awareness training (13%)
- Two-thirds of the organizations studied in a survey now use security AI and automation in their security operations centers, a 10% increase from the previous year. (Cost of a Data Breach Report 2024)
- Organizations that don’t use AI and automation have average breach costs of $5.72 million. In contrast, those that extensively use AI and automation averaged $3.84 million in costs, saving $1.88 million. (Cost of a Data Breach Report 2024)
- Defensive AI is expected to greatly impact cloud, data, and network security domains. (DarkTrace)
- 71% of security stakeholders are confident that AI-powered security solutions are better able to block AI-powered threats than traditional tools. (DarkTrace)
- 69% of enterprise executives believe AI will be necessary to respond to cyberattacks. Most telecom companies (80%) are counting on AI to help identify threats and thwart attacks. (Capgemini Research Institute)
Gartner predicts that AI will positively disrupt cybersecurity in the long term and create many short-term disillusions. Security and risk management leaders must accept that 2023 was only the start of generative AI and prepare for its evolutions. (Gartner Research)
- By 2028, the use of multi-agent AI in threat detection and incident response will increase from 5% to 70% of AI applications, mainly to assist staff rather than replace them. (Gartner)
- Through 2025, the rise of generative AI will lead to a surge in cybersecurity resources needed to secure it, resulting in more than a 15% increase in application and data security spending. (Gartner)
- By 2026, 40% of development teams will routinely use AI-based auto-remediation for insecure code from AST vendors, up from less than 5% in 2023. (Gartner)
AI is a paradigm shift transforming how a society functions. It is an enduring and natural consequence of technological advancement. In recognition of this, most technology executives are focused on experimenting with AI-based technologies. In fact, 9 out of 10 are focusing on platforms such as ChatGPT, Bing Chat, and OpenAI. Additionally, 80% of tech leaders plan to boost their AI investments over the next year. (EY)
Key Takeaways
The state of AI security in 2024 seems to be a bit of a contradiction—global in nature and requiring coordinated efforts. With strict compliance needs, rising geopolitical tensions, and increasingly complex threats, increasing international cooperation is essential to ensuring global business and cybersecurity.
Many organizations find cybersecurity easier to handle now than in previous years. They’re working together better, detecting threats more quickly, and generally have the necessary tools and support.
However, with technology changing so quickly, organizations may find it challenging to decide where to put their focus. While GenAI is widely used in business and within security teams, those who resist might fall behind.
Ignoring generative AI could hinder innovation and lead to the rise of unregulated shadow AI. Instead, develop smart policies for using generative AI that don’t obstruct progress. Moreover, don’t rush into adopting it without understanding the risks.
Resources
- Lakera - GenAI Security Readiness Report 2024
- 25 cybersecurity AI stats you should know - Help Net Security
- Artificial Intelligence Index Report 2024
- https://www.zscaler.com/resources/industry-reports/threatlabz-ai-security-report.pdf
- https://www.immuta.com/resources/the-ai-security-governance-report-introduction/
- State of AI Cyber Security 2024
- https://emt.gartnerweb.com/ngw/globalassets/en/cybersecurity/documents/predicts-2024-cybersecurity-ai.pdf
- Global AI cybersecurity market size 2030 | Statista
- Global concerns about generative AI's impact on cyber 2024 | Statista
- EU AI Act: first regulation on artificial intelligence | Topics | European Parliament
- Artificial Intelligence in Cybersecurity Market Share, Forecast | Growth Analysis & Opportunities [2030]
- https://www.splunk.com/en_us/pdfs/gated/ebooks/state-of-security-2024.pdf
- https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
- https://www.pwc.com/il/en/mc/ai_adopion_study.pdf
- ChatGPT May Already Be Used in Nation State Cyberattacks, Say IT Decision Makers in BlackBerry Global Research
- HiddenLayer’s 2024 AI Threat Landscape Report
- As AI Powered Phishing Gains Steam Organizations Use Password Managers to Combat Threats - LastPass
- Cisco 2024 Data Privacy Benchmark Study
- https://www.immuta.com/resources/the-ai-security-governance-report-introduction/
Learn how to protect against the most common LLM vulnerabilities
Download this guide to delve into the most common LLM security risks and ways to mitigate them.
GenAI Security Preparedness
Report 2024
Get the first-of-its-kind report on how organizations are preparing for GenAI-specific threats.
Free Download
Explore AI Regulations.
Compare the EU AI Act and the White House’s AI Bill of Rights.
Understand AI Security Basics.
Get Lakera's AI Security Guide for an overview of threats and protection strategies.
Uncover LLM Vulnerabilities.
Explore real-world LLM exploits, case studies, and mitigation strategies with Lakera.
Optimize LLM Security Solutions.
Use our checklist to evaluate and select the best LLM security tools for your enterprise.
Master Prompt Injection Attacks.
Discover risks and solutions with the Lakera LLM Security Playbook.
Unlock Free AI Security Guide.
Discover risks and solutions with the Lakera LLM Security Playbook.
Don’t miss the updates!
Subscribe to our newsletter to get the recent updates on Lakera product and other news in the AI LLM world. Be sure you’re on track!
You might be interested
OWASP Top 10 for Large Language Model Applications Explained: A Practical Guide
In this practical guide, we’ll give you an overview of OWASP Top10 for LLMs, share examples, strategies, tools, and expert insights on how to address risks outlined by OWASP. You’ll learn how to securely integrate LLMs into your applications and systems while also educating your team.
LLM Vulnerability Series: Direct Prompt Injections and Jailbreaks
of prompt injections that are currently in discussion. What are the specific ways that attackers can use prompt injection attacks to obtain access to credit card numbers, medical histories, and other forms of personally identifiable information?
Activate
untouchable mode.
untouchable mode.
Get started for free.
Lakera Guard protects your LLM applications from cybersecurity risks with a single line of code. Get started in minutes. Become stronger every day.
Join our Slack Community.
Several people are typing about AI/ML security. Come join us and 1000+ others in a chat that’s thoroughly SFW.
