BLOG

F5 AI Gateway: Solving the Unique Challenges of AI Security and Delivery

Lori MacVittie 축소판
Lori MacVittie
Published November 12, 2024

Over the past year, we’ve learned a lot about generative AI. In particular, we’ve seen best practices for deployment patterns materialize, AI factories becoming a new data center building block, and the concerns about delivery and security of AI models and applications rise to the fore. 

While it’s true that AI applications are the “most modern of modern applications” and heavily rely on APIs, it is also true that AI applications bring unique challenges to the table with respect to security, monitoring, and delivery. 

A significant challenge is due to the non-deterministic nature of AI applications. That is, inbound input and outbound responses vary greatly and evolve over time. AI applications can also pull in data from both structured and unstructured sources that can span multiple locations. Indeed, it was no surprise when we surveyed the market and found a robust mix of public cloud, on-premises, and hybrid deployment plans for AI. 

The trouble with unstructured, unpredictable input and output is the difficulty inherent in determining whether a request or response contains sensitive or incorrect information. While API security and web application firewalls are often used for exactly this purpose, they operate on deterministic content. That is, the format and content of requests and responses are understood and therefore it is easier to detect malicious or sensitive content. With AI, though it leverages the same constructs as APIs and web applications, the content is highly variable, which makes it difficult to craft policies that scan or scrub requests and responses.

Implicit in that challenge is another: AI traffic must be monitored in both directions. This is particularly true given that a significant percentage of organizations plan to leverage AI as a service, such as ChatGPT, Azure, Google, or Hugging Face.   

The security needed to protect AI models

This is why existing API and application security solutions are not enough to protect AI models and applications. AI security is additive to these existing services, offering the additional and unique capabilities necessary to keep companies and customers safe, such as protection against prompt injection, model manipulation, hallucinations, and AI-powered attacks. 

To manage and secure AI models and applications, application delivery and security services must: 

  • Observe and log all requests, responses, and administrative actions to support legal and compliance requirements. 
  • Protect organizations and users by inspecting and acting on malicious, sensitive, or inaccurate prompts and responses.  
  • Accelerate performance using traffic steering policies and controls including volumetric attack prevention. 

Why F5 AI Gateway?

For over 20 years, F5 has risen to the delivery and security challenges posed by shifts in application architectures. AI introduces a new application architecture that, like its predecessors, requires a complementary evolution in application delivery and security. F5 AI Gateway protects, accelerates, and observes AI-powered applications and is a natural progression of application delivery and security for F5. 

That progression includes support for the hybrid, multicloud reality of today’s IT estates, so F5 AI Gateway can be deployed on its own or integrated with existing F5 software, hardware, or services.  

Initially our focus for F5 AI Gateway is on addressing core delivery needs and the OWASP LLM Top Ten items related to real-time security challenges, with more to come in the future.

Additional capabilities such as reporting of a wide array of metrics via OpenTelemetry, careful attention to audit log requirements, semantic caching, rate-limiting, and content-based model routing ensure support for all three AI delivery and security requirements: observe, protect, and accelerate. 

F5 AI Gateway focuses on addressing delivery needs and real-time security challenges identified by the OWASP LLM Top Ten.
F5 AI Gateway focuses on addressing delivery needs and real-time security challenges identified by the OWASP LLM Top Ten.

Recognizing that AI is evolving rapidly, and no two enterprises are the same, F5 AI Gateway further enables organizations to adapt to custom and new requirements with a plug-in ecosystem supported by a software development kit (SDK) for Python, Rust, and Go.   

Securing any app and API, anywhere

F5 is committed to delivering and securing every application and API, anywhere. As applications evolve, that means evolving our portfolio to meet new and unique challenges. When it comes to AI applications, F5 AI Gateway provides what organizations need to deliver and secure them anywhere. 

You can learn more about F5 AI Gateway here or by contacting your F5 account executive or F5 technology partner.