The API management (APIM) market is a crowded space. The latest Gartner Magic Quadrant for Full Life Cycle API Management ranks 22 vendors, with 7 in the Leaders quadrant. Competing in an existing market requires establishing a strong position to differentiate and stand out from the competition. So what makes NGINX’s API management solution different?
At NGINX our goal is to make lightweight, high‑performance software. NGINX Open Source has become the go‑to web server for the world’s busiest sites because its event‑driven architecture scales better than web servers that spawn a process or thread for each connection. NGINX Open Source is also the industry’s most ubiquitous API gateway, an infrastructure component that processes API traffic in APIM solutions like those from Apigee, Axway, IBM DataPower, Kong, Red Hat 3scale, and Torry Harris.
The NGINX Controller API Management Module is a high‑performance APIM solution. In this blog we’ll compare its performance to Kong, a competitor known for its high performance. Kong is built on NGINX, and uses Lua to implement its API functionality, whereas the API Management Module relies completely on native, high‑performance capabilities that are implemented as NGINX Plus modules. NGINX’s API management solution is built on an innovative architecture that decouples the data plane and control plane. All API calls are handled directly by NGINX Plus acting as the API gateway (data plane) without requiring any interaction with the control plane. This results in high‑performance API traffic mediation for both north‑south and east‑west traffic.
A sneak peak at the results: the NGINX Controller API Management Module outperforms Kong by 2x.
A special thank you to Intel for providing the hardware and lab space used to do this testing. Full test and hardware details can be found in the Appendix.
Latency Added to a Single Request
In this test we compared the additional latency introduced by the API manager to a single request, for files of 0 KB and 1 KB in size. We used curl to send the single HTTP request (details).
Kong increases latency compared to the API Management Module: by 44% for a 0-KB file and 24% for a 1-KB file.
API Calls Per Second
A standard HTTP scalability metric is requests per second (RPS). The equivalent metric in the context of APIM is API calls per second. We used wrk to send a continuous stream of requests for either a 0-KB or 1-KB file, on 300 connections for 3 minutes (details).
The API Management Module outperformed Kong – handling 2.6x as many API calls per second for 1-KB responses.
CPU Usage
The API Management Module introduces less latency and handles more API calls per second than Kong because it uses the CPU more efficiently. We measured CPU usage at increasing numbers of API calls per second. We used a single core, so the absolute number of API calls per second is much lower than in the previous test, where we used 22 cores (details).
Kong effectively caps out at 5,000 API calls per second – latency spikes noticeably at call volumes any higher than that, indicating the system is fully loaded. At 5,000 calls per second, CPU usage is 93%, 73% higher than the API Management Module.
API Calls Per Second with JWTs
The final test measures how well each API gateway validates JSON Web Tokens (JWTs), the preferred method for authenticating API requests. Your most important API endpoints will likely be authenticated, so this test more closely approximates a real‑world configuration.
We used the same JWT, signed using HS256, with both gateways (details).
The API Management Module handles more than 2x as many JWT‑authenticated API calls per second as Kong.
How NGINX Controller Works
NGINX Controller is a control‑plane solution that manages the NGINX Plus data plane. NGINX Controller enables you to manage the entire lifecycle of NGINX Plus, as a load balancer, API gateway, or a proxy in a service mesh environment. With NGINX Controller’s API Management Module, you can define, publish, secure, monitor, and analyze APIs.
Under the hood, NGINX Controller generates NGINX Plus configuration which is published to the underlying NGINX Plus data plane. The core configuration loader has a very efficient mechanism for storing configurations in memory, enabling the API Management Module to deliver high performance for APIs.
Conclusion
Many companies already use NGINX Open Source as their API gateway. Capital One has been able to scale to over 12 billion API calls per day using NGINX Open Source. In fact, many of our competitors, including Kong, use NGINX Open Source under the hood in their APIM solutions. Our lightweight design – based on native, high‑performance NGINX configuration along with NGINX Plus modules – enables the API Management Module to scale better than NGINX‑derivative competition.
Latency is one of the most important metrics for end‑user experience. High latency reduces app responsiveness, frustrating users. The API Management Module adds 20–30% less latency to user requests compared to Kong. It also uses system resources more efficiently, using 40% less CPU than Kong for the same workload.
Appendix
Topology
All tests were done using three separate machines connected by 10 GbE links in a simple, flat Layer 2 network.
Hardware Used
The following hardware was used for the testing. All three machines were identical. Hyperthreading was not used. In previous testing we did not observe a major difference in performance from hyperthreading.
| CPU | Network | Memory |
|---|---|---|
| Intel® Xeon(R) CPU E5‑2699 v4 @ 2.20GHz, 22 cores | Intel Ethernet Controller 10‑Gigabit X540‑AT2 | 128 GB |
Software Used
The following software was used to do the testing:
- NGINX Controller API Management Module version 2.
- Kong Open Source version 1.0.0. We did not test Kong Enterprise, but none of its additional features are relevant to our tests (for example, Kong Open Source and Kong Enterprise use the same JWT plug‑in).
curlversion 7.61.0.- Docker version 18.09.1. To make management easier, we ran both Kong and the API Management Module inside Docker containers. We found that Docker reduces performance by about 30%, but because we’re comparing relative performance, the benefits of using Docker outweighed the performance hit.
mpstatin version 12.0.1 of thesystatpackage.wrkversion 4.1.0, installed according to these instructions.
NGINX Configuration
The following NGINX configuration was used for the testing.
We made the following settings to maximize performance. As detailed in the next section, we made similar settings in the Kong configuration.
keepalive_requestsandkeepalive_timeoutwere set to a high number to minimize overhead from setting up TCP connections.tcp_nodelaywas enabled, improving performance slightly by disabling Nagle’s algorithm.access_logwas disabled. Enabling it reduces performance by about 10%.- We used the two
proxy_passdirectives to request the appropriate file size.
Kong Configuration
We added the following configuration directives to kong.conf.default, matching the settings in the NGINX configuration discussed in the previous section.
We ran the following Kong API calls to create a route to the API server. The first command creates a service named test and the second creates the route /test pointing to the server.
The following commands display the configuration of the service and the route, respectively. We pipe the output to the jq tool to make it easier to read the JSON output.
Testing Methodology
Latency Added to a Single Request
We used curl for the single‑request latency testing. To set a baseline, we first made a request to the API server without an API gateway in front of it. We then made the same request with each API gateway in front of the API server to measure the latency they introduced.
The contents of curl-latency.txt:
The chart in Latency Added to a Single Request shows time_total, in milliseconds. Here’s a sample output:
API Calls Per Second
We tested API calls per second with wrk, a scalable benchmarking tool we use often. We tried various combinations of parameters, and this one maximized performance for both the API Management Module and Kong:
The command creates 22 wrk threads (1 per core) and a total of 300 connections across the threads. The -d parameter specifies the duration of the test, in our case 180 seconds (3 minutes). Sample output:
CPU Usage
We tested CPU usage with mpstat (a standard Linux tool for that purpose) while running the wrk command for API calls per second. Sample output (spread across two lines for legibility):
API Calls Per Second with JWTs
We tested JWT performance with the wrk command for API calls per second, with the addition of the -H parameter to insert the JWT in the Authorization: Bearer HTTP header. We generated the JWT and JSON Web Keys (JWKs) using the instructions in this blog post, storing the JWT in a file named test.jwt.
About the Author
Related Blog Posts
Secure Your API Gateway with NGINX App Protect WAF
As monoliths move to microservices, applications are developed faster than ever. Speed is necessary to stay competitive and APIs sit at the front of these rapid modernization efforts. But the popularity of APIs for application modernization has significant implications for app security.
How Do I Choose? API Gateway vs. Ingress Controller vs. Service Mesh
When you need an API gateway in Kubernetes, how do you choose among API gateway vs. Ingress controller vs. service mesh? We guide you through the decision, with sample scenarios for north-south and east-west API traffic, plus use cases where an API gateway is the right tool.
Deploying NGINX as an API Gateway, Part 2: Protecting Backend Services
In the second post in our API gateway series, Liam shows you how to batten down the hatches on your API services. You can use rate limiting, access restrictions, request size limits, and request body validation to frustrate illegitimate or overly burdensome requests.
New Joomla Exploit CVE-2015-8562
Read about the new zero day exploit in Joomla and see the NGINX configuration for how to apply a fix in NGINX or NGINX Plus.
Why Do I See “Welcome to nginx!” on My Favorite Website?
The ‘Welcome to NGINX!’ page is presented when NGINX web server software is installed on a computer but has not finished configuring