HomeProxiesWhat Is Envoy Proxy? The Secret to Microservices Success

What Is Envoy Proxy? The Secret to Microservices Success





“What Is Envoy Proxy?” In today’s microservices-dominated realm, orchestrating traffic flow, equalizing workloads, and bolstering network defenses pose significant challenges. Uncover the capabilities of Envoy Proxy that demystify these tasks, giving you an edge in mastering modern system architecture.

What Is Envoy Proxy?

In the landscape of microservices, an essential component is the network layer 7 (application layer) proxy, which serves as the linchpin in such architectures. In expansive microservices setups, these L7 proxies are crucial as they offer observability, resiliency, and efficient routing. This ensures that external services interact with a seemingly singular service rather than a complex network of numerous microservices.

What Is Envoy Proxy?
What Is Envoy Proxy?

Envoy Proxy stands out in this field as an open-source edge and service proxy renowned for its high performance and lean footprint. Functioning in a similar vein to software load balancers such as NGINX and HAProxy, Envoy was initially developed by Lyft. It has since evolved into a significant open-source endeavor with a vibrant community of contributors. Recognized for its maturity and stability, Envoy Proxy has been embraced by the Cloud Native Computing Foundation (CNCF) and has achieved the status of a Graduated project.

Envoy Proxy: Key Concepts, Functionalities, and Structural Overview

Technical Glossary

To grasp the architecture of Envoy Proxy, it’s important to familiarize oneself with several key terms:

Host This refers to any network-capable entity, like smartphones or web servers. It can also denote a logical network application. A single piece of hardware may host multiple logical hosts, each addressable separately.
Downstream This term describes hosts that initiate connections to an Envoy proxy, sending requests and receiving responses.
Upstream These are the hosts that receive requests and connections from an Envoy proxy and provide responses back.
Listener This is a designated network location where downstream clients can establish a connection. An Envoy proxy maintains multiple named listeners to manage connections with downstream hosts.
Listener A group of upstream hosts to which an Envoy proxy can route traffic. The proxy’s load balancing algorithms decide which member of the cluster handles a given request.
Cluster A group of upstream hosts to which an Envoy proxy can route traffic. The proxy’s load balancing algorithms decide which member of the cluster handles a given request.
Mesh Envoy proxies employ a dynamic configuration system, enabling on-the-fly server setting adjustments without the need for proxy restarts.

Envoy Proxy Architecture

Envoy is a transparent networking layer designed for troubleshooting and managing cloud-native applications. It operates as a standalone executable that integrates seamlessly with any real-world application, regardless of the programming language, and is straightforward to deploy.

As an L3/L4 proxy, Envoy boasts a robust set of filters, facilitating various TCP/UDP proxy functions. It goes a step further by supporting HTTP L7 filters, essential for cloud-native applications, and offers TLS termination capabilities. Envoy excels in load balancing with features like circuit breaking and automatic retries, and it adeptly handles gRPC request and response routing. Its configurations are effortlessly managed via an API, enabling dynamic updates even during runtime.

Envoy Proxy Architecture
Envoy Proxy Architecture

Architecturally, Envoy employs a multi-threaded single-process design. A primary thread oversees coordination tasks, while worker threads are dedicated to processing, filtering, and traffic forwarding. When a connection is initiated, it is assigned to a worker thread for the duration of the transaction.

While Envoy generally operates in a single-threaded mode, it encompasses sophisticated coding to facilitate coordination among its worker threads. For optimal performance, it is recommended to align the number of worker threads with the number of hardware threads available on the system.

Features of Envoy Proxy

Envoy offers a suite of advantages that streamline and enhance the processes of software development and deployment, making them quicker, simpler, more dependable, and more secure. Here’s a concise overview of the essential features that Istio and Envoy software deliver:

Features of Envoy Proxy
Features of Envoy Proxy

Out-of-Process Architecture

The Envoy proxy is engineered to operate in conjunction with every service. Collectively, these proxies create an invisible mesh that facilitates interaction between services, or between a service and external clients, all while remaining oblivious to the underlying network structure. Envoy is compatible with services programmed in any language, including but not limited to Java, C++, Go, PHP, and Python. It seamlessly links services, rendering language-specific communication frameworks obsolete. Moreover, Envoy can be deployed across an entire infrastructure with ease and supports dynamic updates.

L3/L4 Filter Architecture

At the heart of its functionality, Envoy operates as an L3/L4 network proxy, meaning it aids in managing communications within the network and transport layers. Platform teams can leverage Envoy’s integrated filters for a variety of roles, including functioning as a basic TCP proxy, a UDP proxy, an HTTP proxy, or as an authority for TLS client certificate authentication, among others.

HTTP L7 Filter Architecture

Envoy is equipped with an HTTP L7 filter layer, enabling it to support communication at the application layer. Infrastructure and platform teams can utilize these HTTP filters within Envoy’s HTTP connection management system to execute various functions like buffering, routing and forwarding, rate limiting, or even monitoring Amazon DynamoDB activities.

HTTP/1.1 and HTTP/2 Support

Envoy readily accommodates both HTTP/1.1 and HTTP/2 protocols, seamlessly mediating the communication between clients and target servers. Additionally, it is compatible with gRPC, which utilizes the HTTP/2 protocol framework. This allows Envoy to serve as the foundational layer for routing and load balancing of gRPC interactions.

HTTP/3 Support

Beginning with version 1.19.0, Envoy has added support for HTTP/3 protocols, enhancing both upstream and downstream communication capabilities. Envoy also facilitates the translation of communication across various protocols, including HTTP/1.1, HTTP/2, and HTTP/3.

HTTP L7 Routing

In HTTP mode, Envoy provides a routing framework that allows for the redirection of requests based on various parameters like path, authority, content type, and runtime values. This feature proves beneficial for platform teams who aim to employ Envoy as a front or edge API gateway.

Service Discovery and Dynamic Configuration

Envoy utilizes a hierarchical array of dynamic configuration APIs tailored for service discovery, enabling real-time updates pertaining to host data, backend clusters, listening endpoints, HTTP pathways, and security elements. To streamline deployment, the discovery of backend hosts can be conducted via DNS resolution, with the possibility to substitute additional layers with fixed configuration files.

Health Checking

Envoy features health checking capabilities that enable active monitoring of service status within clusters. Based on the insights gained from service discovery and health check information, Envoy intelligently determines the optimal strategy for load balancing.

Advanced Load Balancing

Being a proxy instead of a library, Envoy is equipped to offer sophisticated load balancing capabilities for distributed systems. Its repertoire of advanced traffic control and load management methods encompasses features such as automated retry mechanisms, circuit breaking, mirroring of requests, application of rate limits through an external service, and the identification of outlier instances.

Front/Edge Proxy Support

Utilizing consistent software at the network edge offers significant advantages in terms of observability, manageability, and maintainability, including uniform service discovery and load balancing algorithms. Envoy’s array of features renders it highly appropriate as an edge proxy for contemporary web applications, providing TLS termination, and supporting HTTP/1.1, HTTP/2, and HTTP/3, alongside sophisticated HTTP L7 routing capabilities.


Envoy proxy delivers comprehensive statistics, detailed access logs, and distributed tracing features, enabling Site Reliability Engineers (SREs) to monitor and optimize the performance of the service mesh effectively. It provides metrics for downstream traffic (requests sent by external clients), upstream responses (replies to downstream requests by the proxy), and server operations (handling of client requests) to aid SREs in analyzing network traffic and assessing the performance of the Envoy server. The proxy supports integration with external logging and tracing tools, empowering SREs and infrastructure teams to track, visualize, and understand the call flows within distributed architectures, as well as identify patterns of serialization, parallel execution, and potential latency sources.

Use Cases for Envoy Proxy

Envoy proxy primarily serves two roles: as an adjacent companion to applications within a service mesh (acting as a service proxy), and as a gateway for APIs.

Envoy as a Sidecar

Envoy can function as an L3/L4 proxy, or as a sidecar proxy, to facilitate inter-service communication. Each instance of the Envoy proxy is tied to the lifecycle of the application it accompanies, allowing for the integration of applications from diverse technological backgrounds, including those legacy applications that are not inherently extensible.

Envoy As a Sidecar for Service-to-Service Communication
Envoy As a Sidecar for Service-to-Service Communication

Envoy intercepts all incoming application requests via listeners, such as:

Ingress: These listeners receive requests from other services within the service mesh and channel them to the local application that is paired with the Envoy instance.
Egress: These listeners handle requests originating from the local application that is coupled with the Envoy instance and relay them to other networked services.
The accompanying diagram illustrates the attachment of the Envoy proxy to an application, showing the enablement of communication through both ingress and egress listeners.

Envoy as API Gateway

The Envoy proxy is capable of serving as an API gateway, positioned as a ‘front proxy’ that interfaces between incoming client requests and the backend application. It manages inbound traffic by aggregating request data and intelligently routing it within a service mesh. The diagram below exemplifies Envoy’s role as a ‘front proxy’ or ‘edge proxy’, receiving requests from external networks. When operating as an API gateway, Envoy takes charge of critical edge functionalities, including directing traffic, balancing loads, authenticating users, and conducting edge monitoring.

Envoy as API Gateway for Inbound Traffic
Envoy as API Gateway for Inbound Traffic

Benefits of Envoy proxy

Envoy Proxy, by virtue of its adeptness in abstracting network and security strata, showers manifold boons upon IT echelons including developers, SREs, cloud maestros, and platform brigades. Here are some gems from its trove:

Effective network abstraction

The pioneering out-of-process architecture of Envoy transcends the network layer from the application domain to its own infrastructure bastion. This transmutation propels swifter deployment ventures for application architects, whilst bestowing a centralized dias to orchestrate inter-service communications.

Fine-grained traffic management

Envoy’s allegiance to both the network (L3/L4) and application (L7) realms births a realm of flexible and microscopic traffic steering, embracing traffic bifurcation, steadfast retry protocols, and equilibrium-seeking load balancing.

Ensure zero trust security at L4/L7 layers

Envoy proxy is a knight in shining armor when it comes to fortifying authentication amidst services nestled within a cluster, brandishing robust identity affirmation arsenals like mTLS and JWT. Manifesting authorization at the L7 plateau with Envoy proxy is a breezy affair, ensuring a realm of zero trust. (One may conjure AuthN/Z doctrines with Istio service mesh—the command center for Envoy.)

Control east-west and north-south traffic for multicloud apps

In an epoch where enterprises cast their applications into the boundless clouds, mastering and steering the traffic or dialogues to and fro the data sanctuaries is paramount. With its dual charisma as a sidecar and an API gateway, Envoy proxy gallantly manages the east-west and north-south traffic flows, respectively.

Monitor traffic and ensure optimum platform performance

Envoy unfolds the network mysteries by dispatching statistics, clustered into triads: downstream statistics for incoming requests, upstream statistics for outgoing requests, and server statistics for elucidating the Envoy server specter. Moreover, Envoy sheds logs and metrics that unveil the traffic narrative between services—a boon for SREs and Ops brigades to swiftly spot and mend any performance anomalies.


In wrapping up, we’ve traversed the rich landscape of Envoy Proxy, uncovering its terminology, remarkable features, and robust architecture. Its prowess in abstracting network complexities not only simplifies the life of developers and SREs but also propels applications towards a horizon of enhanced performance and fortified security.

The nuanced traffic management, seamless service mesh integration, and the potent API gateway functionalities are just the tip of the iceberg. Envoy Proxy undeniably emerges as a formidable ally in today’s microservices-driven ecosystem. There’s a realm of possibilities that Envoy unfolds in the network management arena, waiting to be leveraged.

Eager to delve deeper into such transformative technologies? Don’t miss our expansive array of blogs at Twistory, where we continue to demystify the tech orbit, offering you a gateway to informed and strategic technological decisions. Your journey towards mastering modern network solutions is just a click away!

5/5 - (2 votes)


Please enter your comment!
Please enter your name here

Tony Joden

In a fast-changing technological world, he is a guide and a helper for consumers and tech lovers who want to make smart choices by providing detailed reviews that are both informative and trustworthy.

Recent posts