Author Archives: Ant Bullock

When Your Custom AI Becomes the Attack Vector – Why Building Secure-by-Design Matters Just as Much as Speed to Market.

Enterprises across every sector are rushing to build custom AI capabilities; from internal chatbots and AI agents that streamline operations to customer-facing agents that transform service delivery. Yet whilst organisations focus on capturing AI’s transformative potential, many are inadvertently expanding their attack surface and introducing risks that didn’t exist just months ago.

This shift represents a fundamental change in how we must approach cybersecurity. AI innovation is no longer just a technology consideration. It’s become a competitive imperative that directly impacts market position. The organisations that can innovate fastest whilst maintaining robust security will build insurmountable advantages over those that either move too slowly or compromise on security in order to deploy quickly.

The New Threat Vectors

The threats and mitigation strategies outlined below are intended as a top-level introduction. These topics will be covered in more detail at the Secure by Design event by experts from Palo Alto Networks and Unit 42 who work at the bleeding edge of this evolving landscape. Further reading resources are also provided at the bottom of the article for those who want a deeper dive on the full scope of AI-related threats and risk management best practices.

And needless to say, there is no substitute for real-world experience from innovation and security leaders actually addressing these challenges (see a full list of our event speakers here).

Prompt Injection Attacks

Perhaps the most immediate threat facing custom LLM deployments is prompt injection. Attackers craft malicious inputs designed to manipulate model behaviour, effectively bypassing safety controls or extracting sensitive information. Think of it as the AI equivalent of SQL injection attacks.

These attacks can range from simple “jailbreaking” prompts that override model guidelines to sophisticated indirect injections where malicious instructions are hidden in data the model processes. For instance, an attacker might embed invisible instructions in a document that, when processed by an AI system, cause it to leak confidential information.

Mitigation strategies include implementing robust input sanitisation frameworks, deploying context-aware monitoring to detect unusual prompt patterns, and ensuring all AI outputs undergo structured validation before being acted upon.

Autonomous Agent Compromise

As AI agents become more sophisticated and autonomous, they present unique security challenges. To be effective, an agent must have access to company data. Like a human user, with access comes risk. These agents can be compromised through indirect prompt injection attacks or memory poisoning, effectively turning them into malicious insiders with legitimate access to systems and data.

A compromised autonomous agent is particularly dangerous because it operates with the permissions and trust levels of the user or system it represents and does so at software speed. Mitigation approaches include implementing hardware-enforced sandboxing to isolate agent actions, creating segregated data silos that contain only the data required by the agent, deploying behavioural anomaly detection systems, and maintaining immutable audit trails of all agent decisions and actions.

AI-Generated Code Vulnerabilities

Development teams increasingly rely on LLM code generators to accelerate software development, but this convenience comes with significant security trade-offs. Studies indicate that a significant proportion (up to 40%) of LLM-generated code samples contain security weaknesses, including SQL injection flaws, insecure data handling, and hardcoded credential, or other vulnerabilities that make privilege escalation or data exfiltration more likely. Additionally, LLMs can exhibit “contextual blindness”, generating syntactically correct code that’s fundamentally inappropriate for the specific security context or business logic.

The risks extend beyond simple coding errors. LLMs may inadvertently expose sensitive information, reproduce copyrighted code leading to licence violations, or introduce insecure dependencies that expand the attack surface. Over-reliance on code generators can also create a false sense of security, with developers integrating generated code without sufficient understanding or review. As the AI-generated share of code within a code-base increases, this becomes increasingly challenging to manage.

Effective safeguards include mandatory human code review with security focus, automated vulnerability scanning specifically designed for AI-generated code, comprehensive licence compliance checking, rigorous supply chain management for any new dependencies, and developer training programmes that emphasise the unique risks of LLM-generated code.

Excessive Agency and Overautonomy

Perhaps the most subtle but potentially catastrophic risk comes from granting AI systems too much autonomy without adequate safeguards. Simply providing access to more company data as a means to improve performance can be tempting for developers, while the pressure from management to deploy quickly can reduce scrutiny. Overly autonomous agents can make harmful decisions faster than humans can intervene, particularly in high-stakes environments.

Balanced approaches include implementing human-in-the-loop guardrails for critical decisions, designing dynamic privilege scaling that adjusts agent permissions based on risk context, and establishing clear boundaries around what actions agents can take autonomously.

AI Supply Chain Attacks

Modern AI development heavily relies on pre-trained models, third-party libraries, and external datasets. Each component in this supply chain represents a potential attack vector. Compromised foundation models or malicious plugins can introduce vulnerabilities throughout the entire AI stack.

Protection strategies include maintaining comprehensive Software Bills of Materials (SBOM) for all AI components, implementing runtime integrity checks, and establishing rigorous vetting processes for all external AI components.

Training Data Poisoning

When organisations train custom models or fine-tune existing ones, they become vulnerable to data poisoning attacks. Adversaries can inject malicious content into training datasets, corrupting the model’s behaviour in subtle but significant ways. This might manifest as biased outputs, hidden backdoors, or models that behave normally most of the time but exhibit malicious behaviour under specific conditions.

The challenge lies in the scale. Modern AI training involves vast datasets that make manual verification impractical. Effective defences include implementing comprehensive data provenance tracking, using adversarial training techniques to expose models to potential poisoning attempts, and incorporating synthetic data generation to reduce reliance on potentially compromised external sources.

Model Theft and Extraction

Proprietary AI models represent significant intellectual property investments. Attackers can attempt to reverse-engineer these models through sophisticated API querying techniques or by extracting model weights directly. Successful model theft not only compromises competitive advantage but can also expose the organisation to further attacks using their own AI capabilities.

Protection measures include implementing intelligent rate limiting and query monitoring, embedding digital watermarks in model outputs for tracking, and applying differential privacy techniques that add carefully calibrated noise to outputs, making model extraction significantly more difficult.

Making AI Security a First-Class Citizen

The key to navigating these threats successfully lies in treating AI security as a “first-class citizen” throughout the development lifecycle. This means embedding security considerations into every stage of AI development and deployment, from initial design through to production monitoring.

Rather than treating security as a separate checklist to be completed after development, successful organisations integrate security thinking into their AI development process from day one. This requires adopting ‘SecDevOps’ practices specifically adapted for AI systems, where security specialists work alongside data scientists and AI engineers throughout the project lifecycle.

This integrated approach means considering security implications during model selection, incorporating security testing into model validation processes, and designing AI architectures with security as a core requirement rather than an afterthought. It also means establishing clear governance frameworks that balance innovation velocity with robust risk management.

The Innovation Imperative

Here’s the crucial insight: cybersecurity leaders can either be enablers of innovation or bottlenecks. Those who embrace their role as innovation enablers, working collaboratively with AI development teams to build secure systems from the ground up, will help their organisations gain competitive advantage. Those who approach AI security purely as a risk management exercise will find themselves constantly playing catch-up as their organisations struggle to innovate at the pace the market demands.

The most successful enterprises will be those that recognise AI security not just as a defensive necessity, but as a competitive differentiator. The organisations that win in the ‘AI Transition’ will be those capable of developing innovation and security maturity in tandem.

Further Reading

For those seeking to deepen their understanding of AI security frameworks, these four resources provide comprehensive guidance:

Unit 42 (“AI Agents are Here. So are the Threats”) – Unit 42 (Palo Alto Network’s industry-leading threat intelligence, incident response and cyber risk division) draws on insights from analysis of 500 billion daily events to shed light on security vulnerabilities in agentic AI applications, detailing nine attack scenarios and offering mitigation strategies to enhance protection against threats like prompt injection and tool exploitation.

MITRE ATLAS (Adversarial Threat Landscape for AI Systems) – A comprehensive knowledge base of adversarial tactics and techniques against machine learning gen-AI systems. ATLAS provides detailed case studies of real-world attacks and maps them to the familiar MITRE ATT&CK framework, making it invaluable for security teams developing AI-specific threat models.

NIST Artificial Intelligence Risk Management Framework (AI RMF) – The gold standard for AI risk management, offering a structured approach to identifying, assessing, and mitigating AI-related risks. The framework emphasises trustworthy AI principles and provides practical guidance for implementing responsible AI governance across organisations.

OWASP Top 10 for Large Language Model Applications 2025 – The definitive list of security vulnerabilities specific to LLM applications, regularly updated to reflect the latest threat landscape. Each vulnerability includes detailed explanations, attack scenarios, and prevention strategies specifically tailored for development teams building LLM-based applications.

Smart Buildings are no longer futuristic concepts—they’re living, evolving ecosystems. Managing the underlying networks that power them, particularly a Converged Network System (CNS), requires a new mindset. One that’s agile, responsive, and built to adapt to the fast-changing demands of modern facilities.

The Evolving Landscape of Cybersecurity Defence.

Organisations today face an unprecedented challenge in cybersecurity. As digital threats evolve in sophistication and scale, security teams must continuously adapt their defensive strategies to protect critical assets and data. This adaptation has created a fundamental strategic question: is it better to deploy multiple specialised point solutions or adopt an integrated platform approach?

The Point Solution Dilemma

Many organisations have historically built their security infrastructure through incremental adoption of specialised tools, each addressing specific security challenges. According to a recent Gartner survey of 162 large enterprises, organisations use an average of 45 cybersecurity tools. *

Supporters of the ‘best-of-breed’ viewpoint site the following reasons:

• Specialised Capabilities: Point solutions can provide superior performance in their specific domains
• Flexibility: Organisations can select the exact tools that match their unique security requirements
• Innovation Adoption: Teams can quickly implement cutting-edge solutions as they emerge

However, this approach has created significant challenges for security teams:

Integration Complexity

Tools with different interfaces, protocols, and requirements create integration nightmares, causing substantial time and resource strain on IT teams. These disconnected systems can result in security gaps between tools that attackers can exploit.

Data Overload

Multiple security tools generate overwhelming volumes of alerts and data, much of it disconnected. Security teams struggle to correlate information across platforms, potentially missing critical security incidents while drowning in noise.

Budget Inefficiency

Investing in numerous specialised tools often results in organisations overspending on multiple cybersecurity licenses, often with overlapping capabilities, that provide diminishing returns or become redundant as organisational needs evolve.

Skills Fragmentation

Each point solution requires specialised knowledge to deploy and maintain effectively, spreading technical expertise thin and creating potential security blind spots when key personnel are unavailable or leave the organisation.

The Platform Approach Alternative

The single-vendor – a.k.a. ‘platform’ – approach to cybersecurity, introduced by vendors in the late 2010s, continues to gain momentum. It’s prominently featured in Gartner’s “Top Cybersecurity Trends for 2025” as Trend 4: Cybersecurity Technology Optimisation.

For CISOs and CTOs focused on operational efficiency, cost control, and effective incident management, a platform approach offers compelling advantages:

Reduced Complexity

A unified platform consolidates multiple security functions into a single ‘pane of glass’ and management framework. This simplification reduces the risk of human error and provides a clearer operational picture for security teams.

Seamless Integration

Purpose-built platforms offer natively integrated components designed to work together. This designed-in integration enables automation of repetitive tasks and provides comprehensive visibility across the entire infrastructure, enhancing the effectiveness of security measures.

Unified Response Capabilities

With a platform approach, organisations can implement consistent security policies and rapidly respond to emerging threats across their entire digital estate. Threat intelligence flows seamlessly between components, enabling faster detection and response.

Advanced Analytics Through Centralisation

By centralising security data collection and analysis, platforms can more effectively leverage AI and machine learning to detect anomalies, predict threats, and automate responses – capabilities that are difficult to implement across multiple disconnected point solutions.

Long-Term Cost Efficiency

While point solutions might appear cost-effective initially, their cumulative expenses often escalate due to licensing fees, maintenance requirements, operational overhead and specialised training needs. A unified platform can significantly reduce long-term costs by streamlining operations and consolidating tool management.

Furthermore, with the right vendor, platform solutions are not an all-or-nothing proposition. They are an à la carte offering where allowing organisations to pick and choose the components required for their specific circumstances by activating only the licensing elements they require.

Finding Your Optimal Security Strategy

The decision between a platform approach and point solutions isn’t binary. Many organisations benefit from a hybrid strategy that leverages the strengths of both approaches based on specific security domains and organisational requirements.

Key considerations for your strategy include:

• Assessing your current security architecture and identifying integration pain points
• Evaluating the operational burden of managing multiple tools versus potential capability trade-offs
• Understanding your organisation’s threat profile and specific security requirements
• Considering your security team’s size, expertise, and operational capacity

As the Head of AI Readiness at Ideal, I am witnessing firsthand how organisations are grappling with one of today’s most pressing cybersecurity challenges: the explosive growth of generative AI in the workplace. Through my many conversations with CISOs, DPOs, and IT leaders, a clear pattern has emerged – enterprises are struggling to balance the transformative potential of AI with its inherent security risks.

The challenge is stark: employees are increasingly turning to third-party Large Language Models (LLMs) and generative AI applications that exist beyond traditional security perimeters. While these tools offer unprecedented productivity benefits, they also create new vectors for data exfiltration and security breaches that keep security leaders awake at night.

The challenge is stark: employees are increasingly turning to third-party Large Language Models (LLMs) and generative AI applications that exist beyond traditional security perimeters. While these tools offer unprecedented productivity benefits, they also create new vectors for data exfiltration and security breaches that keep security leaders awake at night.

From Wild West to Lockdown

In my discussions with security leaders, I’ve observed two posture extremes.

Some organisations have adopted a ‘Wild West’ stance, placing no restrictions on AI tool usage. While this is great for innovation, it leaves sensitive data vulnerable to exposure through uncontrolled AI interactions. At the opposite end of the spectrum, some have implemented complete lockdowns on AI applications. This approach is bad for both innovation and security because it often backfires by driving employees toward ‘Shadow AI’ (the use of unauthorised AI tools).

The solution lies in finding the middle ground through what we call the ‘Permit, Restrict, Monitor’ (PRM) framework. This balanced approach allows for controlled innovation while maintaining robust security measures.

Many of the security professionals I speak with admit – off the record at least – that they only have a partial picture of AI risk in their organisations. Conducting a comprehensive AI audit can establish a baseline. For example, a “NIST AI Audit” maps out in detail the current risk landscape and provides gap analysis. This allows risk and security teams to prioritise their mitigation activities and build a comprehensive risk management framework. Even building an inventory of AI usage, using existing proxy analysis and network monitoring tools, is a good start.

Building a Balanced PRM Security Framework

The key to successful AI governance starts with establishing clear boundaries. Forward-thinking organisations are implementing enterprise agreements with AI providers that include specific data protection clauses. These agreements ensure that enterprise data is properly ringfenced and, crucially, cannot be used for training future AI models.

But agreements alone aren’t enough. Successful implementations I’ve seen include:

• Deployment of secure API gateways that regulate data flow between approved AI applications and internal systems
• Deploying runtime security solutions continuously observe AI application behaviour, including system calls, file access, network connections, and data flow
• Implementation of strong Data Loss Prevention (DLP) solutions that can identify and block sensitive data from being shared with unauthorised AI platforms
• Network-level controls that restrict access to non-approved AI services while maintaining smooth access to authorised tools
• Where the organisation has remote or hybrid workers, providing access to permitted AI applications via a secure enterprise browser (e.g. Palo Alto’s Prisma Access Browser) that allows data exfiltration risk and Shadow AI use to be monitored and controlled

Making It Work: The Human Element

Perhaps the most critical element of an effective PRM AI security strategy is employee engagement. The most successful organisations are those that have invested in the right AI tools for their employees and in comprehensive training programmes that go beyond simple dos and don’ts. These programmes help employees understand not just how to use approved AI tools safely, but also why using those tools is both good for them and for the organisation.

Crucially, this balance between the self-interest of employees and the best interests of the enterprise, will only be effective if the permitted AI apps provided by the organisation are best-in-class and provide all the core functionalities they require. If not, employees simply use workarounds with Shadow AI.

In a recent survey of 6,000 knowledge workers in the US, UK & Germany, nearly half believe that Generative AI (GenAI) will improve their job promotion prospects and 46% would continue to use their preferred AI apps, even when banned by their organisations. Of knowledge workers using Shadow AI, a third do so because their organisation does not provide a permitted alternative. In this sense, when it comes to GenAI, investing in the right tools is a pre-requisite of an effective data security posture.

Monitoring for Success

Effective monitoring is crucial for maintaining security without hampering productivity. Leading organisations are implementing

• Real-time usage analytics to understand how AI tools are being utilised across the enterprise
• Regular security audits that specifically target AI-related vulnerabilities
• Compliance tracking systems that ensure adherence to both internal policies and external regulations

If you are wondering whether your permitted AI policy is working effectively to curb Shadow AI use, a good rule of thumb is that if your usage analytics show significantly less than 75% of your users are using your permitted apps, then it’s likely that they are using non-permitted alternatives. (75% GenAI usage rates among knowledge workers seems to be a consistent figure from multiple surveys over the last year.)

Looking Forward

The reality is that AI is not just another technology trend – it’s a fundamental shift in how work gets done. Organisations that try to completely restrict its use will find themselves at both a competitive and security disadvantage, while those that embrace it without proper controls risk serious security breaches.

The path forward is clear: implement a balanced framework that enables innovation while maintaining security. This means having clear policies about which AI tools are approved for use, permitting access to secure, best-in-class tools, establishing strong technical controls to enforce these policies, and maintaining robust monitoring systems to ensure compliance.

As AI technology continues to evolve, so too must our security frameworks. The organisations that will thrive are those that view AI security not as a barrier to innovation, but as an enabler of sustainable AI adoption.

With a fully automated warehouse, Wi-Fi coverage and reliability are king.

Ideal has just won a tender for the design, implementation and support of a Wi-Fi installation in a fully automated warehouse.

For the client, this is their first step into a fully automated forklift pick and place. The plan is to dedicate half of an existing warehouse to full automation as a prototype towards a potentially larger system in the future.

The launch of DeepSeek R1 has stunned Silicon Valley, launched global counter-intelligence initiatives and crashed tech shares on Wall Street. But that’s not what CISOs should be worrying about.

By Pax Zoega, Head of AI Readiness, Ideal.