AI Factories Produce the Most Modern of Modern Apps: AI Apps

Modern applications have undergone significant evolution, driven by the need for agility, scalability, and enhanced user experiences. The advent of cloud computing has marked a pivotal point in this evolution, providing the foundation for highly distributed systems that can scale efficiently and operate seamlessly across multiple environments. Application modernization is the evolution to API-based systems, which drove architecture to be hybrid and multicloud. 

AI applications further the reliance on APIs and establish hybrid and multicloud as the new normal. Containerization technologies have further revolutionized application deployment by enabling consistent and isolated environments for software regardless of the underlying infrastructure. These advancements are not only pivotal for traditional modern applications but also serve as the backbone for AI factories. The agility provided by cloud computing and containerization are essential for managing the intensive computational demands of training and deploying AI models.

Orchestration tools like Kubernetes are essential for managing containerized applications at scale. Kubernetes automates the deployment, scaling, and management of these applications. In AI workflows, Kubernetes coordinates the deployment of AI workloads across multiple systems, efficiently allocating resources and enabling seamless scaling of AI models.

A critical aspect of this process is MLOps, which addresses the unique challenges of developing, training, and operationalizing AI models. MLOps supports continuous integration and delivery (CI/CD), ensuring that AI models are continuously tested, updated, and deployed. Additionally, the adoption of microservices architectures has transformed modern application development by breaking down monolithic systems into smaller, independently deployable services. This approach accelerates CI/CD, leading to faster and more reliable software updates.

Security has always been a cornerstone in the evolution of modern applications. With the rise of sophisticated cyber threats, modern applications have integrated advanced security measures such as web application firewalls (WAF), distributed denial-of-service (DDoS) protection, and API security. It's crucial to distinguish between Layer 3/4 DDoS attacks targeting network and transport layers and Layer 7 DoS attacks aimed at the application layer. 

As AI progresses, the threat model evolves, especially for large language models (LLMs) accessed through natural language processing (NLP) interfaces. Risks to LLMs include vulnerabilities like those highlighted by the OWASP Top 10 for LLM applications, such as prompt injection attacks and data extraction threats, along with model-specific DoS attacks. These security advancements ensure AI applications remain resilient and secure, maintaining the integrity and reliability of AI-driven solutions against evolving threats.

If you’ve been following our series on AI factories, we’ve explored topics like API security, network segmentation, traffic management, retrieval-augmented generation (RAG), and data processing units (DPUs). Let’s take a step back and discuss how an AI factory fits into the F5 AI Reference Architecture, how AI factories produce AI applications, and the requirements to deliver and secure them.

Drawing on our nearly three decades of experience helping customers deliver high-performing applications, F5 developed an AI Reference Architecture that includes seven building blocks (inference, RAG, RAG corpus management, fine-tuning, training, external service integration, and development) and four deployment types (AI-SaaS, cloud-hosted, self-hosted, and edge-hosted). For more information on these, visit our blog defining an AI factory. As AI applications are the most modern of modern applications, connected with APIs and extremely distributed, this reference architecture addresses performance, operations, and security challenges essential to delivering AI applications. 

Two of the most common building blocks for AI factories are training and inference; however, each building block is required to make a fully functional AI application. Some organizations may build an AI factory to handle functions required for training or inference. Others may rely on third parties to consume services for model training or provide infrastructure for inferencing.

AI applications at the core are distributed, potentially running on distributed Kubernetes container platforms. This is the very definition of a modern application and from where our phrase, “an AI application is simply the most modern of modern applications” comes from. But what does this mean for enterprises building AI factories? AI applications need the same delivery and security services of today’s modern applications such as Kubernetes ingress, multi-tenant egress, DNS-based traffic management, resiliency, and observability. For security, DDoS protection, web application and API protection, and optimization technologies are required. 

While it may seem surprising that we focus on delivery and security features common to both AI and modern applications, rather than solely on AI-specific security concerns, this approach is intentional. AI applications do introduce unique risks, but often, the basics of application security and delivery are overlooked in favor of new, innovative features. By addressing shared security and delivery needs first—what we call the low-hanging fruit—organizations can make immediate improvements using their existing expertise in app security. Although AI applications may involve new components, they still rely heavily on web and API interactions, which are well within the capabilities of current NetOps and SecOps teams. By starting with proven security and delivery strategies, teams can build a solid foundation and gradually tackle the unique challenges posed by AI applications.

The F5 AI Reference Architecture highlights not only AI factory output but also challenges and risks to making the AI application successful for the organization’s business objectives. To do this, we decided to use a pair of complementary top 10 lists of risk and challenges: the OWASP Top 10 for LLM Applications and the F5 Application Delivery Top 10. We have mapped each component and interconnection of the building blocks and outlined where the applicable OWASP LLM Top 10 and the F5 Application Delivery Top 10 apply. In the case below, see an example for AI Inference.

Every component and interconnection point within an AI application benefits from the well-established delivery and security protections designed for modern applications. This isn’t to suggest that these conventional protections are solely sufficient for securing AI applications. By leveraging these proven delivery and security solutions as a foundation, organizations can achieve a significant enhancement to their AI applications’ delivery and security posture. From this solid starting point, teams can progressively develop specialized skills and adopt the advanced tools necessary to tackle the unique challenges and risks that AI applications present.

Right now, the F5 AI Reference Architecture is available as a preview. Customers will have the opportunity to explore the full AI Reference Architecture in an interactive experience starting late next month at AppWorld, F5’s premier application security and delivery conference.