API Protection for AI Factories: The First Step to AI Security

Earlier in our AI factory series, we defined an AI factory as a massive storage, networking, and computing investment serving high-volume, high-performance training and inference requirements. Like traditional manufacturing plants, AI factories leverage pretrained AI models to transform raw data into intelligence. AI apps are simply the most modern of modern apps, heavily connected via APIs and highly distributed.

We start with the premise that an AI app is a modern app, with the same familiar challenges for delivery, security, and operation, and that AI models will radically increase the number of APIs in need of delivery and security. 

So, mission accomplished! We got this, right? We know how to securely deliver modern apps. We have well-established application security best practices and understand the intricacies for protecting APIs. We have solutions that provide comprehensive protection and consistent security across distributed environments.

Not so fast! Hang on to your GPTs as we briefly dive into the details.

While AI apps share the same risks and challenges as those for modern apps (both of which are essentially API-based systems), AI apps have specific implications—training, fine-tuning, inference, retrieval augmented generation (RAG), and insertion points into AI factories.

So yes, AI apps, like modern apps, have a highly distributed nature, create tool sprawl, and complicate efforts to maintain consistent visibility, uniform security policy enforcement, and universal remediation of threats across hybrid and multicloud architectures. In addition to grappling with this “ball of fire” of risk and complexity, defenders must contend with the random nature of human requests and the highly variable nature of AI-procured content that can lead to unpredictable AI responses, as well as considerations for data normalization, tokenization, embedding, and populating vector databases.

Bringing together data to AI factories creates a holistic and connected enterprise ecosystem, and the integration and interfaces between of all these moving parts requires a robust API security posture.

The rise of API-based systems from rapid modernization efforts led to a proliferation of architecture across data centers, public clouds, and the edge. This has been chronicled by F5’s annual research, including our 2024 State of Application Strategy Report: API Security | The Secret Life of APIs report.

The complexity of hybrid and multicloud architectures is intensified by AI apps. Organizations are rapidly assessing and diligently employing governance strategies for a litany of AI apps and underlying AI models, ultimately deciding on an AI-SaaS, cloud-hosted, self-hosted or edge-hosted approach, sometimes on a use case and per-app basis. For example, some use cases involve training in the data center before production—when inferencing is performed at the edge.

The connectivity into these models and associated services is largely facilitated through APIs. For example: connections into popular services for inferencing. AI apps bring an exponential rise in API usage, as a single API may ultimately have thousands of endpoints, and API calls can be buried deep inside business logic, outside the purview of security teams. This leads to untenable complexity and serious implications for risk management.

It turns out cloud did not eat the world, but APIs did. And AI is going to eat everything. As described in our blog post, Six Reasons Why Hybrid, Multicloud Is the New Normal for Enterprises, a highly distributed environment provides flexibility for deployment of AI apps and associated services that ensure performance, security, efficiency, and accuracy. This is especially important for data gravity concerns. It’s no surprise that 80% of organizations run AI apps in the public cloud and 54% run them on-premises. Regardless of the approach, every facet of the architecture must be secured, and protecting APIs is the first place to start. 

AI applications are dynamic, variable, and unpredictable. Safe adoption of AI apps requires greater emphasis on efficacy, robust policy creation, and automated operations. Inspection technologies will play a critical and evolving role in security.

For API Security within AI apps and ecosystems, in particular, organizations will need to offset the increased likelihood of unsafe usage of third-party APIs by architecting security for outbound API calls and related use cases, in addition to the inbound security controls that are largely in place. Bot management solutions cannot rely on “human” or “machine” identification alone, given the rise of inter-service and East-West traffic, for example, from RAG workflows. And since the supply chain of LLM applications can be vulnerable, and impact the integrity of training data, models, and deployment platforms (which are often sourced from third parties and can be manipulated through tampering or poisoning attacks), security controls need to be implemented everywhere—in the data center, across clouds, at the edge, and from code, through testing, into production.  

AI Gateway technology will play a key role in helping security and risk teams mitigate risks to LLMs such as prompt injection and sensitive information disclosure, but the first step in the AI security journey begins with securing APIs that connect AI apps and AI ecosystems by making security intrinsic throughout the AI app lifecycle.