Scaling containers is more than just slapping a proxy in front of a service and walking away. There’s more to scale than just distribution, and in the fast-paced world of containers there are five distinct capabilities required to ensure scale: retries, circuit breakers, discovery, distribution, and monitoring.
In this second post on the art of scaling containers, we’ll dig into circuit breakers.
Circuit Breakers
Thomas Edison, who famously invented thousands of gadgets that didn’t work and a few famous ones that did, gave us the concept of a circuit breaker back in an 1879 patent application. Yes, patents were a thing even then. While Edison’s version used fuses, which must be replaced (some of us might remember frantically searching for them in our old timey cars), more modern versions are built to “trip” and stop the flow of electricity. They can then be reset, restoring normal flow and operation.
In the context of scale, circuit breakers operate on the same principle. They detect an “overflow” and deliberately cut it off to avoid overwhelming services on the other end of the connection. They can also be reset, subsequently restoring normal flow of requests and responses.
Circuit breakers have been a part of load balancing proxies for quite some time. The premise has been that if – after X tries – you still can’t reach a given service, it’s out of commission. There’s reason to keep asking it for something it can’t give you, and doing so is only wasting resources at the proxy and the network. So after a (typically) configurable number of failures, a proxy will “break” the circuit and refuse to attempt further connections.
This is not the same as a retry, though the process appears similar. Retries operate on the premise that the request will eventually succeed. A circuit breaker operates on the premise that the request will fail, and thus wasting time and resources doing so is to be avoided.
Once the problem has been resolved, the circuit breaker can be “reset” and normal flow can resume.
In the early days, this process was manual. An operator was required to perform the reset after assuring that the target service was indeed back in service. In more recent years, this process has become automated through the use of health monitoring. This typically includes periodic attempts to reach the service and, upon success, resets the circuit breaker to allow normal operations once again.
Circuit breakers are particularly important in a containerized, microservices setting because of the high volume of traffic flowing to and from and between services. While some failures may be recognized quickly, others will not be noticed until a lengthy TCP timeout due to issues in the network stack. Timeouts incur undesirable latency, so circuit breaking and retries should take into consideration overall application tolerances (or intolerance, as the case may be) for latency. Configuration of such values needs to take into account timeout values and the business’ expectations with respect to performance. Low tolerance of latency may require fewer retries and faster circuit breaking behavior.
About the Author
Related Blog Posts
F5 accelerates and secures AI inference at scale with NVIDIA Cloud Partner reference architecture
F5’s inclusion within the NVIDIA Cloud Partner (NCP) reference architecture enables secure, high-performance AI infrastructure that scales efficiently to support advanced AI workloads.
F5 Silverline Mitigates Record-Breaking DDoS Attacks
Malicious attacks are increasing in scale and complexity, threatening to overwhelm and breach the internal resources of businesses globally. Often, these attacks combine high-volume traffic with stealthy, low-and-slow, application-targeted attack techniques, powered by either automated botnets or human-driven tools.
F5 Silverline: Our Data Centers are your Data Centers
Customers count on F5 Silverline Managed Security Services to secure their digital assets, and in order for us to deliver a highly dependable service at global scale we host our infrastructure in the most reliable and well-connected locations in the world. And when F5 needs reliable and well-connected locations, we turn to Equinix, a leading provider of digital infrastructure.
Volterra and the Power of the Distributed Cloud (Video)
How can organizations fully harness the power of multi-cloud and edge computing? VPs Mark Weiner and James Feger join the DevCentral team for a video discussion on how F5 and Volterra can help.
Phishing Attacks Soar 220% During COVID-19 Peak as Cybercriminal Opportunism Intensifies
David Warburton, author of the F5 Labs 2020 Phishing and Fraud Report, describes how fraudsters are adapting to the pandemic and maps out the trends ahead in this video, with summary comments.
The Internet of (Increasingly Scary) Things
There is a lot of FUD (Fear, Uncertainty, and Doubt) that gets attached to any emerging technology trend, particularly when it involves vast legions of consumers eager to participate. And while it’s easy enough to shrug off the paranoia that bots...